How we cook GitHub, Jira and Jenkins with Domino

Why would I use it?

I'll describe briefly how we use GitHub, Jira and Jenkins together to make better development process. More details will come later when I get some desire :). OK, lets look what benefits we have from using these tools.

GitHub with Jenkins

GitHub has number of Service Hooks and one of them is for Jenkins. It allows to trigger build jobs when pushes are made to GitHub. You don't need to have any skill to do that, let's look on screen below. So once you made a push, Jenkins start to work.

GitHub with JIRA

It's possible to made connection between JIRA and GitHub, so once you made push to GitHub JIRA will update related ticket, you only need to specify ticket number in comments. As a result you will have history with commits directly in JIRA ticket. Let's look on ticket I've closed few days ago.

You can see what exactly have been done in order to fix the ticket. Also you can get more details if you wish, you only need to click on any of those updated files and then you will see that

Jenkins and Domino

That is most complicated part. In order to synchronize changes from GitHub to Domino application we use Jenkins (if you remember once push happens we trigger hook on Jenkins). Unfortunately there are no plugins that could do that, instead you have to write your own. That's it why we have ~60 seconds delay (I mentioned it in previous posts) after push.

Summary

For me JIRA and GitHub is de facto standard our days. Jenkins in our case is necessary to use in order to push changes from GitHub to Domino. It's a bit complicated process so you need before, if it is really what you need.

Other articles in this series

1. How we build our web applications based on Domino
2. Split back-end and front-end areas, they should not block each other and be independent as much as it is possible.
3. Front-end guys should not have any knowledge about Domino, they don't need IBM Designer installed at all.
4. Back-end guys must have knowledge about Domino, however they don't need to use IBM Designer, only in very rare cases. Backend should be done using only Java (no LS/SSJS/@Formula etc)
5. Using Git+Jira is must and Jenkins server as builder server
6. Auto-tests.

Java as backend for web applications based on Domino

Is it enough to use only Java?

Yes. Our newly created applications have been written only for web and we did all back-end with Java only (the only exclusions are selection for Views). No, we do not miss @Formula, Lotus Script or SSJS, we simply do not need those languages in our daily work. Java is better than LS/SSJS for us. You can develop much better/faster using Java and get all candies: open sources, forums, stackoverflow or even shift at some point to Java completely if it has sense for you. However, if you read my main posts I mentioned that we still had some problems while work with Java, Solution isn't really perfect and let me explain all those weakness we have right now.

• Domino has 2 Java runtimes (huge pain for us): one is for xPages and another one for Java Libraries+Agents. Java Libraries can be reuse in xPages and back JAVA/JAR are not accessible from Libraies/Agents. Our Java libraries are core for xPages and it means we have to build JAR from them and include them for xPages. It takes time and we really do not like that, it simply looks wrong. The solution is to put JAR to Domino OS, however we are limited with access to Domino's OS. Due to that fact we have one serious disadvantage - our JAR files that we attached to Libraries and Agents have to be extracted and loaded to memory each time we run agents.
• We are forced to use IBM Designer (and few of us would like to work in different IDE). As I mentioned we work via GitHub and it takes some time to build project (developer -> github -> jenkins -> domino). That's not a problem for FE, as they can do their tasks locally and after all just commit changes. BE part is located on Server so if we work without IBM Designer we have to wait 1-2 mins till changes come to it and only then we can do our tests. That's sad.

Does it mean @formula, Lotus Script, SSJS are not required to know?

Those languages are still important part of our 'old applications' and it will take years to completely re-write them to web and there should be a sense to do that, as it is time/money, @formula language is still only 1 way to make selection in Notes Views.

Pros

• open source;
• problems resolving (forums, community);
• more abilities;
• in case of migration to java platform, easy migration;

Cons

• 2 Java Runtimes
• ...

Summary

We are happy with what we have so far. We still have problems in terms of how to setup our development process. We will continue to look for perfect solution for our problems.

Other articles in this series

1. How we build our web applications based on Domino
2. Split back-end and front-end areas, they should not block each other and be independent as much as it is possible.
3. Front-end guys should not have any knowledge about Domino, they don't need IBM Designer installed at all.
4. Back-end guys must have knowledge about Domino, however they don't need to use IBM Designer, only in very rare cases. Backend should be done using only Java (no LS/SSJS/@Formula etc)
5. Using Git+Jira is must and Jenkins server as builder server
6. Auto-tests.

Frontend developers without knowledge about Domino

Do FE developers must have understanding of IBM Domino/Notes platform?

I believe they should know we use Domino as server, however that's it. Domino is server, so it is responsibility of BE developers. Few years ago when we had mixed UI and Backend in our projects each time FE developer had to do something in IBM Designer I saw such face.

I do not like such face :), one more reason why we forced our self to do what we did.

More details how FE developers works

I will try to describe workspace for typical FE developer on our project. First of all as I mentioned before they do not need IBM Designer, so they can use whatever they prefer. As all FE are located in GitHub after you import project you will get that (example from my Eclipse)

Once they commited anything to GitHub special mechanism would be triggered on our Jenkins build server and it will push all changes to Domino application.

Except many obvious benefits, any disadvantage?

Atm only 2 disadvantages:

1. It takes ~60 seconds after you commit appear in Domino application (Jenkins server has to push all changed elements from GitHub to Domino)
2. If you lost connection to internet - it's a problem (no access to GitHub)

Summary

If you do that you could get such benefits as: GitHub, freedom for FE, better mood for FE, new FE don't need to know anything about IBM Domino

Other articles in this series

1. How we build our web applications based on Domino
2. Split back-end and front-end areas, they should not block each other and be independent as much as it is possible.
3. Front-end guys should not have any knowledge about Domino, they don't need IBM Designer installed at all.
4. Back-end guys must have knowledge about Domino, however they don't need to use IBM Designer, only in very rare cases. Backend should be done using only Java (no LS/SSJS/@Formula etc)
5. Using Git+Jira is must and Jenkins server as builder server
6. Auto-tests.

Split backend & frontend when build web applications

Is it really a problem to mix Backend (BE) and Frontend (FE)?

Yes, it is a problem, it's simply wrong. Some of you will argue with me and say:

• I'm doing both: FE and BE and I'm 'pro' in both areas.
• Our FE developers learned Domino platform in few weeks, so we have no problems, they feel comfortable with doing stuff with IBM Designer.
• Our FE guys simply say what to change and we do that, they send us some template each time we need to change something.
• We are doing internal website and our internal employee do not care about speed, modern UI etc.

I believe some of you are really perfect in both areas: FE and BE and you have enough time to improve skills in both areas just in time, so you don't follow practice that 10 years old. Honestly, at some point I thought I was good enough in both areas (FE and BE) to do modern stuff, however more I worked with web more I realized that my skills FE is not good enough to build really modern, fast and easy maintained web application. I spent years before I gave up and said to myself: I'm quite good in BE and I'm not bad in FE and that's actually enough, after that life became a bit more easy :).

Where is a problem?

Let's take an example when new FE developer joint to the team and he got tasks to change some UI.

1. Classic example (mail9.ntf, form memo)

Problems:

• almost impossible to deal with that if you are not skilled in Domino;
• have to know @Formula language to change <Compute Values> or hidden formulas;
• have to know that fields might have some properties (classes, styles, id etc);
• should ask for help each time they need to change UI;

2. xPages example (discussion9.ntf)

Problems:

• more skill required about Domino, Java, SSJS and xPages especially;
• controls will generate HTML automatically and it is a problem for our team as we aiming for 100% control;
• as it's even more complicated, potentially FE will ask to involve BE developers each time they need to chagne UI.

If we split BE and FE then what?

Then life become much easier :), application will get better structure, things will go faster. How to do that? In our case we found template framework The Apache Velocity Project and tested it enough to make sure it would cover our needs. Then we started to move out UI from logic step by step, and yes, that process took months but once we did it we started to receive benefits immediately. Today we keep UI (HTML templates, CSS, JS) as files and our FE guys use GitHub. They are free to create new templates and organize their structure. They have huge freedom (compare to what they had 2 years ago) of how to do their job. There are really a lot of documentation + stackoverflow.com that can help. However Velcoity itself is quite intuitive and simple framework.

Wanna see 'Hello world' example?

In example bellow we pass 2 items to template (HashMap<String, String> map equal to Dim mylist List as String in LotusScript) then Velocity engine does rest of work and return result. In our project we simply pass all fields from document to template so FE guys have freedom and ability to work independently. It's really easy, proven by our FE guys.

HashMap<String, String> map = new HashMap<String, String>();
map.put("name", "Velocity");
map.put("author", "Joker");
velocity.applyVelocityTemplate("hello.world", map)

+++

## Velocity Template; id 'hello.world'
<html>
    <body>
       ## our hello world example
       Hello $name World!
       $author
    </body>
</html>

||

\/

<html>
    <body>
       Hello Velocity World!
       Joker
    </body>
</html>

Summary

Split and manage!

Other articles in this series

1. How we build our web applications based on Domino
2. Split back-end and front-end areas, they should not block each other and be independent as much as it is possible.
3. Front-end guys should not have any knowledge about Domino, they don't need IBM Designer installed at all.
4. Back-end guys must have knowledge about Domino, however they don't need to use IBM Designer, only in very rare cases. Backend should be done using only Java (no LS/SSJS/@Formula etc)
5. Using Git+Jira is must and Jenkins server as builder server
6. Auto-tests.

How we build our web applications based on Domino

Intro

I've been working with Domino for many years (however very possible less than many of you :-), I started to work with version 5 at the beginning and in few months we migrated to 6.X). I worked in few companies with absolutely different projects and processes of development. There were lot of 'hell projects' (without any structure inside, just mess) and few really 'cool projects' where I could learn something. I always tried to bring the best of the old projects into new. Now I'd like to describe how we develop applications in Domino today. No revolution in our approach, however I believe that is quite good one. The only required thing in that process - you must have time and desire to change process.

Team and projects

We are small and quite typical team: few back-end and front-end developers, manager and no QA as we do not do mistakes :). Roll out each 2 weeks with new features. We are developing CMS that manage external websites of our company.

Few very important points about websites we are doing here:

• website's pages have to load extremely fast (means 200-400 ms to load page);
• clean HTML, all (!) tags, attributes, javascripts should be controlled;
• easy roll out from development environment to production environment without any interrupt for users.

Process of development

Now let's look on most important points of development process. I'll describe each point more deeply in upcoming posts this week. Possibly I will add more points during that time, lets see.

1. Split back-end and front-end areas, they should not block each other and be independent as much as it is possible.
2. Front-end guys should not have any knowledge about Domino, they don't need IBM Designer installed at all.
3. Back-end guys must have knowledge about Domino, however they don't need to use IBM Designer, only in very rare cases. Backend should be done using only Java (no LS/SSJS/@Formula etc)
4. Using Git+Jira is must and Jenkins server as builder server
5. Auto-tests.

We've done almost all the list, some areas require some improvements however concept works and now it's only question of time. All front-end developers do not use IBM Designer in their daily job, back-end developers still use it (rarely), everything goes to GitHub, and then Jenkins take care about rest. That image illustrate what we have.

You are welcome to give your comments, idea, suggestions or something negative about that, everything can help and improve or process and wait for new posts soon :)

Other articles in this series

1. How we build our web applications based on Domino
2. Split back-end and front-end areas, they should not block each other and be independent as much as it is possible.
3. Front-end guys should not have any knowledge about Domino, they don't need IBM Designer installed at all.
4. Back-end guys must have knowledge about Domino, however they don't need to use IBM Designer, only in very rare cases. Backend should be done using only Java (no LS/SSJS/@Formula etc)
5. Using Git+Jira is must and Jenkins server as builder server
6. Auto-tests.