Tuesday, August 06, 2013

Split backend & frontend when build web applications

Is it really a problem to mix Backend (BE) and Frontend (FE)?

Yes, it is a problem, it's simply wrong. Some of you will argue with me and say:
  • I'm doing both: FE and BE and I'm 'pro' in both areas.
  • Our FE developers learned Domino platform in few weeks, so we have no problems, they feel comfortable with doing stuff with IBM Designer.
  • Our FE guys simply say what to change and we do that, they send us some template each time we need to change something.
  • We are doing internal website and our internal employee do not care about speed, modern UI etc.
I believe some of you are really perfect in both areas: FE and BE and you have enough time to improve skills in both areas just in time, so you don't follow practice that 10 years old. Honestly, at some point I thought I was good enough in both areas (FE and BE) to do modern stuff, however more I worked with web more I realized that my skills FE is not good enough to build really modern, fast and easy maintained web application. I spent years before I gave up and said to myself: I'm quite good in BE and I'm not bad in FE and that's actually enough, after that life became a bit more easy :).

Where is a problem?

Let's take an example when new FE developer joint to the team and he got tasks to change some UI.

1. Classic example (mail9.ntf, form memo)
  • almost impossible to deal with that if you are not skilled in Domino;
  • have to know @Formula language to change <Compute Values> or hidden formulas;
  • have to know that fields might have some properties (classes, styles, id etc);
  • should ask for help each time they need to change UI;
2. xPages example (discussion9.ntf)
  • more skill required about Domino, Java, SSJS and xPages especially;
  • controls will generate HTML automatically and it is a problem for our team as we aiming for 100% control;
  • as it's even more complicated, potentially FE will ask to involve BE developers each time they need to chagne UI.

If we split BE and FE then what?

Then life become much easier :), application will get better structure, things will go faster. How to do that? In our case we found template framework The Apache Velocity Project and tested it enough to make sure it would cover our needs. Then we started to move out UI from logic step by step, and yes, that process took months but once we did it we started to receive benefits immediately. Today we keep UI (HTML templates, CSS, JS) as files and our FE guys use GitHub. They are free to create new templates and organize their structure. They have huge freedom (compare to what they had 2 years ago) of how to do their job. There are really a lot of documentation + stackoverflow.com that can help. However Velcoity itself is quite intuitive and simple framework.

Wanna see 'Hello world' example?

In example bellow we pass 2 items to template (HashMap<String, String> map equal to Dim mylist List as String in LotusScript) then Velocity engine does rest of work and return result. In our project we simply pass all fields from document to template so FE guys have freedom and ability to work independently. It's really easy, proven by our FE guys.
HashMap<String, String> map = new HashMap<String, String>();
map.put("name", "Velocity");
map.put("author", "Joker");
velocity.applyVelocityTemplate("hello.world", map)
## Velocity Template; id 'hello.world'
       ## our hello world example
       Hello $name World!
       Hello Velocity World!


Split and manage!

Other articles in this series

  1. How we build our web applications based on Domino
  2. Split back-end and front-end areas, they should not block each other and be independent as much as it is possible.
  3. Front-end guys should not have any knowledge about Domino, they don't need IBM Designer installed at all.
  4. Back-end guys must have knowledge about Domino, however they don't need to use IBM Designer, only in very rare cases. Backend should be done using only Java (no LS/SSJS/@Formula etc)
  5. Using Git+Jira is must and Jenkins server as builder server
  6. Auto-tests.


Roman Kopac said...

This is getting interesting. I hope you will share some more details.

I wonder why do you use Domino at all. I guess there are other CMS's out there that are more suitable for this kind of workflow. Is it ACL, NoSQL store or something other that convinced you to use Domino?

Dmytro said...

@Roman Kopac

I wonder why do you use Domino at all.
> because of:
- we still have applications that are done in classic way, (and as I mentioned it'd take a lot to redesign them);
- many nice features;
- easy replications, most easy I've ever saw in my life.
- easy scalability, almost about copy database.

> other CMS
very possibly, but we have our own CMS that does exactly what we need, so no reasons to look for another solutions.

Nick Wall said...

I have been using angular.js and knockout.js to separate back from front end and for ui creation. Did you asses at any js frameworks? If yes, did you find any benefits using Velocity?

Dmytro said...

@Nick Wall

Good question, happy you asked.

We have to compute all HTML as much as we can for each page so it is ready for user. Reason is SEO. Once 'google search bot' will make request to our pages - we would be able to return all HTML with all TEXT inside it. Than google bot would have data to index (very very important thing for us).

So benefits is SEO in our case.

Andriy Kuba said...

The key word is "Split" but "Domino"

Domino is so good as others and so bad as others. Generally IBM move it to "yet one java EE server".

We a try to build our application in a clear way to comfort work for all developers.

>I have been using angular.js and knockout.js to separate back from front end and for ui creation. Did you asses at any js frameworks?

As we could see it almost all about fronted. More of then it suppose to use some mixes, that could be complicated. Would be good if you will write some article about you experience with angular.js

Velocity is completely clear - just few rules, easy to configuration and simple to use. It could completely divide business logic and client view.

We even start to look on Mustache, but we yet not agreed about logic-less.

Karsten Lehmann said...

We have the same kind of setup for the same reasons - no XPages for the UI, but Domino database for the backend with a clear separation.

For the web frontend, the UI is generated either via XAgent or a classic servlet in an OSGi plugin. In our case, the template engine is Jamon ( http://www.jamon.org/index.html ).

Dmytro said...

@ Karsten Lehmann

nice to know we are not alone :), never heard about Jamon, will look briefly, so far looks good, I think it can almost do same what Velocity.

Unknown said...

Great stuff and great comments.

Nick Wall said...

@Andriy Kuba. Not sure if "...experience with angular.js" comment was aimed at me...either way, this is a great topic of discussion...

Our app is accessed via "normal" browser, but also "consumed" by other devices, some off-line. All our data is surfaced as JSON, the app is built as a SPA (Single Page Application). When App was first developed, we didn't use a js framework...big mistake. If we had used framework, and only used framework for 2 way data binding, we would have saved ourselves many, many, many hours of frustration.

For anyone interested, a great intro to Angular.js is here:

the-ntf said...

HashMap map = new HashMap();
map.put("name", "Velocity");
map.put("author", "Joker");
velocity.applyVelocityTemplate("hello.world", map)

If, as is implied by your article, you are using a Document to source the contents of the map, then you should use the OpenNTF Domino API. Because then you could just do...

velocity.applyVelocityTemplate("hello.world", document)

There'd be no need to implement the interim map, as long as the item names match the map keys you want to use.

If Velocity allows it in the tokens, you could even use @formulas as your keys and they will be evaluated against the document during a .get(key).

(Well, this is in my current branch, not the OpenNTF release.)

Dmytro said...

@ Nathan Freeman

you are right, we can and we do something very similar, we pass all items with same name but map.
however if we pass document then we have to call method from it every time we need to read value (doc.getItemValueString('itemName')) and if you use map then it will be map.itemName.

+ I've to look on 'OpenNTF Domino API' never tried it. Will do short tests tomorrow, thanks!

Domino Interface said...

This has been an interesting post. We developed a similar approach but is not as sophisticated as yours. We use a single XPage as a front-end HTML container. The front and back end is totally decoupled. We created our own front-end framework using Dojo though I have been toying with Backbone.js. JSON REST-like calls are using to retrieve and send data to the server. All data is stored as JSON in our domjDB structure which is similar to MongoDB.

We find the approach much better than the pure XPages approach which I find limiting. There is a clear separate between the data and form which is not true for traditional XPages. I rarely use the Domino Designer since we developed our own front-end compiler which we are planning to convert it into a web-based tool. This allows non-Domino developers to create apps.

Andriy Kuba said...

@Nick Wall. "Our app is accessed via "normal" browser, but also "consumed" by other devices, some off-line. All our data is surfaced as JSON, the app is built as a SPA (Single Page Application)."

Good solution then. We need to generate full page on the server, because of speed and because of search engine an so on. That's why we do not use client side frameworks hardly.

General idea still the same.
Raw Data -> Business Logic -> Clean Data -> UI.

"Clean Data" is the JSON that you send from Server to the Client. UI representation could be generated as on the server (trivial application) as on the client (single page application) in this case. Mustache would allow to do it completely the same way for both sides.

We are moving to this, we want to have part of pages that is generate don the server and part that is dynamically builds on client side. In the same time we want to work with both of them in the same manner.

Andriy Kuba said...

@Nathan Freeman. "velocity.applyVelocityTemplate("hello.world", map)"

As @Dmytro Pastovenskyi said we already use it in some places.

We still thinking about this in general.

Document could have some items that is not allowed to be shown on web. Templates could be edited people that have lower level of access then core developers have. This is a problem.

So this way need to be used with caution. Possible ways: divide documents on "secure\not secure", create filters to protect put some items in to the map, give all users that works with template more trust :)

John Eggers said...

nice piece

John Eggers said...

Really awesome article regarding backend for frontend, will really help many developers out there.