Webwork and Stripes: A Simple Performance Comparison
Well, it took a bit longer to run the load tests than i thought (due to finishing some work for the next build of the site), but we finally got around to it yesterday.
Like I mentioned in the previous post..I decided to rewrite one of our pages in Stripes, and perform a load test to compare the Stripes version to the WebWork 2.2.2 version.
To briefly recap the last post..the ONLY differences in the 2 versions of the page are the Action class and the jsp. Thats it. All other layers...Spring, iBatis, data model, etc are the EXACT same in both versions. Even the business logic within each Action class is the exact same. The only diff is the each Action has a different superclass, and Webwork is configured with xwork.xml, and Stripes uses annotations. Thus most all of the application code (non-framework) is the same. As for the jsp..the webwork version uses webwork tags exclusively, and the Stripes version uses a combination of Stripes form tags, and JSTL. Since most of the application code is the same, i believe any differences in test results should be due to the differences in Webwork and Stripes.
We ran the load test on a test server, not a real prod server. I know the final numbers wouldnt mimick live numbers, but it is still a valid as a relative comparison between the 2 versions.
I'll be honest...after reading about performance issues with OGNL, and seeing how some of the framework code looks..i was kind of expecting Stripes to be a bit faster. From working with the two..that was just my gut feeling.
However, I will admit I was pretty surprised by the results. In our tests, the Stripes version of the page was 3-4 times faster than the webwork version! Thats a pretty significant difference.
Now..i'm not trying to say the Stripes is always that much faster than webwork, or that in all cases Stripes will be faster (though i do indeed believe Stripes will usually be faster in most cases)..this test was just comparing the 2 frameworks on 1 page in our site. So make of this test what you will. I'm just reporting what we found in our case..nothing more. It is good ammo for me to use to push for officially using Stripes in our application. Since the site has only been complete for 6 weeks, its a little early to propose a big architecture change. However, i plan to convert the site over in my free time as I can...the sooner I do it, the less will have to be changed later.
Having worked with webwork since December, I feel i have a good handle on how to develop with it. I really dont have any big complaints, either, as I enjoyed using it. As good as webwork is, I just think Stripes is one step better. If you can get the benefits of the self-documenting nature of annotations (and no config files), a little speadier development, and a performance boost..I think you have to consider Stripes a serious player in your web framework decision.
I hope people continue to discover how easy Stripes is to use and the user base continues to grow. I think this framework really deserves the attention.
As I continue to migrate the site (unofficially) from webwork to stripes, i will post any problems i run across, as well as any "this is awesome" moments, too. I expect there to be a lot more of the latter.

7 Comments:
Can you post some more details about the test? How did you time it? What were the times? Is this a load test or just the timing of one page? Is one hit to a stripes page 3-4x faster than one hit to the same webwork page? Or do the performance increases only show up at a higher load?
Looking forward to more "Stripes is awesome" moments :)
were there Sitemesh used in the systems? if yes, would it be possible to make a comparison with-without sitemesh?
Nameless: sitemesh is not used on out site.
Paul: I didnt run the test..was just told the results. If i understand it correctly...30 req per second were spun off for 90 seconds or so. So the same number of requests were made in total. The Stripes version of the page finished 3.5 times quicker than the Webwork version. it seems that as more requests are made, Stripes continues to handle the load better without slowing down..whereas webwork seemed to peg the cpu much quicker than Stripes did.
I ran a performance comparison of Stripes, Struts 1, Struts 2 (WW2). The same basic app was used for all tests, being a simple nested form displayed as text fields using the provided taglibs (stripes:text, nested:text, and s:text).
I was surprised by the results:
- Stripes 47 hits/s (127 hits/s with no return page).
- Struts1 102 hits/s (180 hits/s no return page)
- Struts2 3 hits/s (102 hits/s with no return page).
Yes 3 hits/s for Struts2, Not a typo! The test was run on a fairly low spec machine (P1.6) but still...
I was very much looking forward to the reusable templating in Struts 2/WW2 but unless performance issues can be sorted out in a hurry I will be deciding between Struts 1 or JSF (yet to be tested) for my next project.
PS: I got the same results using WebWork2.
When I switched to Velocity themed templates (instead of the default Freemarker templates) it improved to around 11hits/s. Closer inspection using JProfiler shows OGNL is using up all the resources.
Note: that I don't actually use OGNL in my test JSP. However, Webwork appears to use OGNL internally for all it's setting and getting of request parameters.
IMO, there are going to be some unhappy people who migrate from Struts 1 to Struts 2.
thanks for the numbers Frankie. Sounds like your test also indicated the possible performance issues with OGNL in webwork/struts 2.
I do wonder why/how struts 1 could be faster than stripes, though.
Today I realised I messed up the tests by running them in debug mode. I reran the tests and also bumped up the JVM memory. This is probably a better reflection although until I can run them on a high end Unix server I won't know for certain.
Technology - Hits per second with 1 user / 10 users:
Struts 1 - 109 / 191
Stripes - 88 / 140
WW2/SAF2 with default FreeMarker templates - 12 / 7
WW2/SAF2 with Velocity templates - 22 / 15
JSF - 27 / 40
For a while I was starting to believe the press that we have actually made progress over the last few years.... In reality we have only gotten better at creating Hello World applications.
Post a Comment
Links to this post:
Create a Link
<< Home