Friday, June 15, 2012

Grails 2.0 vs ASP.NET MVC 4 (Conclusion)

This is the conclusion to two previous posts that considered the pros and cons of using Grails 2.0 vs ASP.NET MVC 4 for open source projects.  I have used both technologies for a few years apiece.  However, their latest incarnations have offered benefits over their respective predecessors.  Therefore, it was due time for me to reevaluate.

If you read my last post on this topic, ScrumTime - Grails 2.0 vs ASP.NET MVC 4, you saw that I came up with seven questions to use for the comparison.  I answered each question to the best of my ability at the time.  Since that post, I have researched both frameworks with two different open source projects: ScrumTime and BumbleBee.

ScrumTime source code on GitHub:
BumbleBee source code on GitHub:
If you are bored and actually want to view the source of all four projects, you will notice that the project with the most active work is the BumbleBee Grails 2 implementation.

What is BumbleBee?
BumbleBee allows project managers to track the progress of large software integration projects.  It provides a single view of Mantis bugs, test scripts, unit tests, design documents, and third party vendor issues.  The default security is currently single-sign on with active directory.  The SSO with AD is a post all its own since it is implemented in a java web stack.

What happened to ScrumTime?
I have only put ScrumTime to the side for the moment because I have a business "need" for the BumbleBee application immediately.  It seemed like an excellent candidate to finish my evaluation of Grails vs MVC.

What is the final web-stack decision and why?

I decided to complete the BumbleBee project in Grails 2.0 because of many factors:

  • velocity of implementation is quicker for me due to less source code and the following
    • repository finders are dynamically generated automatically (MVC does not have this)
    • pre-request, post-request, and post-view filters are much simpler to implement than MVC.
    • the default in-memory database supports live changes to the domain models without a server restart (MVC does not have this)
    • nearly all aspects of the web application can be changed without requiring a server restart (MVC has limited support for this)
  • the end result is a war file that will run on any hardware that can run jdk 6+ (MVC can only run on windows based computers (Mono would require significant EF changes)
  • more database vendor options after the product is developed
    • Because Grails is based on Gorm which is built on Hibernate, changing the back-end database vendor after the entire application is complete does not pose a challenge.  This can be done by changing the properties in the DataSource.groovy file.  However, the story with MVC is pretty bad.  The only options in MVC are to use nHibernate or create your own ORM from scratch.  Even in these cases, there is no automatic integration between the model objects of the MVC application based on nHibernate as there is with model objects based on the Entity Framework.  I can only guess that Microsoft prefers to make it easy for developers to connect to their own databases and not so easy to connect to other vendor databases.
These factors were too important to the success of an open source project for me to ignore.  

I hope to complete the first release of BumbleBee within the next few weeks.

Wednesday, May 2, 2012

"NoSQL" Databases - (MongoDB)

I have recently become a big fan of the "NoSQL" databases.  When I first read about them, I thought that the entire concept seemed a bit like fantasy land because of its simplicity and elegance.  It seemed too good to be true.  However, there are already thousands of world-class applications using these very real databases.  In fact, companies like Netflix, Facebook, and Twitter (to name a few) make extensive use of NoSQL.  These three alone make use of Apache Cassandra in particular.

ThoughtWorks has an excellent introduction to NoSQL databases that I recommend to anyone that wants to learn more about the subject.  The most important question to ask of any new tool or technology is: 'How is this thing going to help me do my job better or faster?'  

Taken directly from the ThoughtWorks post...

Why would you want to use a NoSQL database?
One of fundamental drivers is that you have challenges in your business that are difficult to solve using traditional relational database technology. If you have an excellent relational model running on a mature database that provides all the features you need then there is probably little need to change your data storage mechanism. Here are some use cases where it is sub-optimal to use a conventional database: -
  • Your relational database will not scale to your traffic at an acceptable cost
  • Your data is supplied in small updates spread over time so the number of tables required to maintain a normal form has grown disproportionally to the data being held. Informally if you can no longer print your ERD on an A3 piece of paper you may have hit this problem or you are storing too much in a single database.
  • Your business model generates a lot of temporary data that does not really belong in the main data store. Common examples include shopping carts, retained searches, site personalisation and incomplete user questionnaires.
  • Your relational database has already been denormalised for reasons of performance or for convenience in manipulating the data in your application.
  • Your dataset consists of large quantities of text or images and the column definition is simply a Large Object (CLOB or BLOB).
  • You need to run queries against your data that do not involve simple hierarchical relations; common examples are recommendations or business intelligence questions that involve an absence of data. For the latter consider "all women in Paris who do have a dog and whose ex sister-in-laws have not yet purchased a paperback this year" as a contrived example, "all people in a social network who have not purchased a book this year who are once removed from people who have" is a real one if you want to target advertising on a site that says "Fred bought X".
  • You have local data transactions that do not have to be very durable. For example "liking" items on websites: creating transactions for these kind of interactions are overkill because if the action fails the user is likely to just repeat it until it works. AJAX-heavy websites tend to have a lot of these use-cases.

There are two of these bullet points that I find interesting in the context of ScrumTime 1.0.  The next release of ScrumTime will have extensive support for attaching artifacts to nearly all object types.  For example, a Release may have a Statement of Work document uploaded and associated with it.  Also, a Story may be associated with a few photographs of a whiteboard discussion for which it relates.  Therefore, the ScrumTime dataset will consist of large quantities of binary objects.  Also, the collaborative nature of ScrumTime is an excellent fit for AJAX as was shown in release 0.9.  The 1.0 release will use AJAX in a similar way that Facebook and Google use it to promote team collaboration and communication in so far as indicating another user is online.

Due to the fact that the purpose of ScrumTime is to provide a usable open source agile project management tool to the masses, it stands to reason that providing a simple setup and configuration to get running quickly is a high value requirement.  Therefore, when considering the use of both a relational database and a NoSQL database to solve all of the data storage needs of ScrumTime, I have to think about the impact to the guy or girl that deploys ScrumTime.  If that person has to setup two databases, they are not going to be happy with me.  I could write a setup application to configure both databases, but that effort takes away from adding value to the ScrumTime product itself.  Therefore, if all of the features of ScrumTime may be implemented in one database, all the better.

In the past week, I have implemented the ScrumTime domain models in a NoSQL implementation called MongoDB.  I also tried CouchDB and RavenDB.  I like all three of them, but there is no standard querying syntax between them.  This had me wondering which of the three would be the best choice for ScrumTime since none of them seem to follow a standard convention.  But, what is a standard convention...well, it is usually dictated by the most popular choice.  As a result, I decided to research the number of open jobs on the market in the United States that mentioned either of the product names.  The winner by far was MongoDB which tells me that it is currently the most popular.  I was also able to find numerous options for language specific database drivers and lots of documentation for MongoDB.  So, MongoDB is the database choice for ScrumTime 1.0.

If you have not had a chance to look at NoSQL, please make time.  It is not all hype.

Friday, April 20, 2012

ScrumTime - Grails 2.0 vs ASP.NET MVC 4

In my previous post, I documented the process of considering the various "Web Stacks" that are widely available for the implementation of ScrumTime 1.0.  This post is a continuation of that post.

Here are the factors that go into considering these two "Web Stacks" as viable open source platforms:

  1. Does the platform support multiple databases in such a way that does not require a major refactoring for each database supported?
  2. Does the platform have native support for a multilingual user interface?
  3. Does the platform support a modular view solution that can support various screen sizes and capabilities?
  4. Does the platform allow the business logic layer to be decoupled and reused in other non-web application capacity?
  5. Is there a large development community that will be able to look at the code base to add to the project at a future date?
  6. Is the platform significantly cheaper to develop and deploy than the competition?
  7. Is the platform stable over a long period of time? (do security patches to the OS threaten the features of the deployment?)

Last week, I began implementing ScrumTime 1.0 in ASP.NET MVC 4.  The source may be found here.  This week I began implementing ScrumTime 1.0 with Grails 2.0.  The source may be found here.

Here are the answers to the questions above:

Grails Answers

  1. Yes. Grails uses hibernate under the covers.  Therefore, Grails has support for all major database vendors and the JDBC drivers are widely available.
  2. Yes. Grails has a large focus on multilingual user interfaces with the inclusion of the i18n folder with each newly created project.  This folder holds the resource files that contains the different strings for the different languages supported.
  3. Yes. Grails supports a concept of "layouts" and "templates" that allows a developer to create and reuse view components.
  4. Yes. Grails compiles down to a set of Java classes that may be referenced from any other Java application running on the same VM.
  5. No. Grails is built on Groovy which is a dynamic language that compiles into byte-code that runs on the Java virtual machine.  Unfortunately, the number of Groovy developers in the world are far outnumbered by the number of C, C++, VB, C#, and Java developers.
  6. Yes. Grails is much cheaper to develop and deploy.  It is conceivable that the cost of development and deployment is free since it can be done with open source tools and platforms.
  7. Yes. Since Grails runs in a JVM, there is little to no relationship between the JVM and the operating system.  
ASP.NET MVC 4 Answers
  1. Yes, well sort of. MVC 4 supports the latest Entity Framework 4.1 that includes something known as "Code First".  Code First support all Microsoft databases, Oracle, MySQL, and supposedly any database that has an ADO.NET provider.  Another option is to use NHibernate instead of EF at all.  NHibernate supports all of the major database vendors.
  2. Yes. MVC 4, like all .NET applications, may contain "resource files."  There are several options for making use of the resource strings, but the most clear option is to use a reference in the model's constraint annotations.
  3. Yes. MVC 4 allows developers to define "layouts" and "partial views" that follow the view reuse best practices of other web frameworks like Grails and Ruby on Rails.
  4. Yes. The most straightforward way to implement a reusable Business Logic Layer in MVC 4 is to implement the BLL in a separate .NET DLL.  Then the DLL may be used by the MVC 4 web application.
  5. Yes. MVC 4 may be built on .NET VB or .NET C#.  The usage of C# seems to be growing versus VB according to job trends.  Therefore, in the comparison with Grails, I have focused on MVC 4 .NET C#.  The C# open source community is very large.  It is not quite the size of Java or C/C++, but it is global.
  6. No. The cost of development is basically free if Visual Studio 2011 Express is considered sufficient (I have been using Visual Studio Professional).  The cost of deployment is likely more than depends on the amount of data that is required to be stored and the number of users that would be connecting. 
  7. No.  The .NET framework is affected by system updates to the Windows operating system.  While it is possible to separate the .NET specific updates from the rest of the operating system updates, that is a hassle.  Also, due to the success of the .NET framework for Microsoft over past ten years, it has become more intertwined with the operating system.  As a result, it is not always clear where .NET begins and and the operating system ends.  Therefore, at the very least, ASP.NET MVC 4 requires Windows operating system knowledge and research for each patch deployment...that is an ongoing cost that is arguably greater than that of maintaining a JVM. 

Based on the criteria above, the ASP.NET MVC 4 "Web Stack" seems to be the prudent choice because of the large development community.  However, the development cost and deployment cost is not as positive. By April 23, 2012 June 15, 2012, I will make the final decision and proceed with the full implementation at that time.

*Update (04/23/2012) - ASP.NET MVC 4 is the first to be heavily researched.  The source may be found here on GitHub. 

*Update (05/14/2012) - Grails 2.0 heavy research begins.  The source may be found here on GitHub. 

Monday, April 16, 2012

The "Best" Web Stack for ScrumTime

I remember as an intern at IBM in Charlotte in 1992 that the computer science community debated what was the "best" desktop operating system.  Is it Windows 3.1 or IBM OS/2?  My office desktop at the time was an OS/2, yet I had a Windows 3.1 at home and in the labs at Appalachian State University.  To me the "best" was whatever operating system provided most growth of my knowledge and the greatest potential of growth for my career.

I believed Windows 3.1 was the "best" because I had heard it was the best and it was more widely available.  However, I learned while using OS/2 at IBM that it was superior to Windows 3.1 in terms of various technical capabilities.  Once I felt that I knew both operating systems down to a medium level, I determined that there was only one question remaining.  'Which operating system will provide the greatest potential growth of my career?'  A long story short, I left IBM in 1995 well before Windows 95 was released.  I haven't touched OS/2 since.

What is the point?  I learned a lesson about the computer industry at that time that is still relevant today.  

'Never assume that one product or solution is better than another simply because others (including the media, advertising, your cousin's bowling league, etc.) say that it is better.  Always, treat every new technology with an open mind and take the time to learn it as much as time permits.  After doing so, decide which is better for your immediate application and for your needs in the foreseeable future.  But, never commit to only one technology because it is a complicated world.'  
While working on ScrumTime (off and on) during the past five years, I have learned three web technology stacks: Ruby on Rails, Grails (java based), and ASP.NET MVC.  All three have been very beneficial  to my career.  Ruby on Rails is a fantastically concise, clean, and stable web development experience.  However, it has never had the developer following that a Java or a Microsoft solution has had.  Therefore, I have not pursued Ruby on Rails with ScrumTime or with my career.

Grails is basically a knockoff of Ruby on Rails.  However, it was being pushed in the Java community around 2007 and 2008 which is why I decided to learn it.  As a result, releases 0.4, 0.5, and 0.6 of ScrumTime were implemented in Grails.  I am a huge fan of Grails and I would like to see more Java developers implement projects Groovy and Grails.

ASP.NET MVC is not as concise and clean as either Ruby on Rails or Grails, but it has a huge following in comparison.  Once I realized that it was open source, I decided to implement ScrumTime 0.9 in ASP.NET MVC 2 hoping that ScrumTime users and developers would find it useful.  There has been more interest in ScrumTime since switching to the Microsoft stack, but I suspect it is because more features were made available in 0.9 release than in any prior release.

After trying Grails and ASP.NET MVC implementations of ScrumTime (and other applications), I do not claim that one is the "best".  However, I can say that one fits the needs of ScrumTime better than the other.  I used the following points to make this assessment.

  • Using a language or platform that is known to a large number of developers is important to keeping an open source project relevant over the course of time.
    • indicates that the need for Java developers is about the same as that for .NET developers.
      • When combining words "java scrum" vs ".net scrum" the winner seems to be Java.
      • Also, combining "java web" vs ".net web", Java is the winner here too.
    • indicates that the need for ASP.NET MVC developers is greater than the need for Grails developers.
    • This point is effectively a draw between ASP.NET MVC and Grails simply because both are still very new technologies.
  • ScrumTime should be easy to deploy on all popular operating systems.
    • The Java platform may be deployed on whatever operating system you like with greater ease than the Microsoft ASP.NET platform.
      • The mono mvc project has potential but it does not seem to be mainstream among ASP.NET developers.  Open source solutions that are not easy to deploy are not often used.  
    • Grails applications deploy as war files which are the standard for all Java based web application deployments. 
    • These indicate that Grails wins this point.
  • The Groovy language is more concise than all existing .NET languages that support MVC. 
    • At first glance, this sounds like a subjective point.  However, I can prove that the number of lines to implement ScrumTime in Grails is fewer than the number lines required to implement ScrumTime in ASP.NET MVC.
    • What is the point? 
      • Developers that are new to a project or technology will more quickly ramp up when the total breadth of a project is smaller rather than larger.
      • Like Ruby, Groovy is very expressive and conveys only what is intended within a convention.  The downside is that the developer must understand the convention in order to clearly understand the source code.  However, the convention is the same across all projects regardless of the author.  As a result, the developer will quickly understand the source for all other projects in the future.  ASP.NET MVC also follows convention over configuration to a point.  However, it is simple to fall-back to non-convention following practices with ASP.NET MVC.  As an initial developer, that is an awesome feeling to throw anything in wherever I choose.  But, it is not so great for the developer that must support the product.
    • Grails seems to win this point.
When considering these points, it seems that ScrumTime 1.0 should be implemented in Grails.  However, the story does not end with ScrumTime 1.0.  The plan is that ScrumTime 1.0 will have the ability to support mobile browsers and even offer a JSON/XML interface to native clients such as iPhone, Android, and desktop applications. 

These mobile and desktop native clients will most likely use the ScrumTime 1.0 RESTful interface and they will have their own release cycles.  It is my hope that other developers will be inspired to create their own open source projects to implement such clients.  If not, I may get around to it some day.

I am planning to begin the project with two in Grails 2.0 and the other in ASP.NET MVC 4.  It seems like a waste of time, but I think it will be good research and I plan to document my findings.

Wednesday, April 11, 2012

Scrum as It Relates To Billing

I have found that the term "feature" in the context of scrum can be a bad word among purest. Many software engineers that practice scrum use words such as user story, task, points, actor, sprint, iteration, chicken, pig, etc. However, if you are software engineer that has had the pleasure of working directly with your customers, you have found that they do not necessarily know what you are talking about and they probably should not need to know. Therefore, the term "feature" rears its ugly head in order to bridge the gap between your jargon and the language with which they are already familiar. I say ugly because many purest view the term "feature" as a throw-back to the waterfall model. But, I claim that a feature is really the high-level manifestation of a collection of user stories.

For example, suppose the customer considers the user login as a feature while the following user stories make up the functionality of that feature from the software engineer's point of view:
  • As an anonymous user, I must login to the system in order to be granted the correct access rights within the system.
  • As an authenticated user, I want to be able to log off the system because my computer is shared with other users.
When speaking directly to my customers, I generally map the user stories to "features" in my head or on paper for their benefit. However, it becomes easy to lose track of this mapping in more complex scenarios especially when dealing with multiple customers at one time. Regardless, all of my customers want to be billed based on satisfactory delivery of "features" and not user stories (if you have had a different experience, please let me know your secret). It has been my wish for several years to work through a solution that would bridge the gap between the impression that the customer has of the product in terms of billing with the impression that the software engineers have in terms of creating the product.

This solution is to be included in ScrumTime 1.0 as a new level in the existing work item structure. ScrumTime 1.0 will have support for features in the form of children of the product work item level. For example:
  • Product - Personal Inventory Tracker
    • Feature - User Login
      • Story - As an anonymous user, I must login ...
        • Task - Create the login div container
Also, the feature may be assigned to a release but not to a sprint. I consider "release" to be another one of those terms that is important to the customer but not important to the software engineer. For example, a customer may say that they want a specific feature to be in a specific release. From the software engineer's point of view, the set of user stories that make up that feature must be implemented by the end of a specific sprint. In fact, it may take two or more sprints to complete all of the stories that make up the feature. This sort of visibility must be available in ScrumTime 1.0 or I will have no use for it in my day-to-day work. Yes, I plan to use ScrumTime 1.0 day-to-day.

One of the main benefits that I look forward to is that task work hours will now bubble up to the level of features providing a actual billable hours for the customer. In the past I have had to rely on my own custom spreadsheets to keep track of the mapping and the correct hours to charge. Based on my experience, I will need the ability to credit hours at the feature level in the event that a task takes more time than should be billed.
This topic leads to an entirely new aspect of the ScrumTime road map known as the planned customer view. The idea is to allow your customers to be able to login to your hosted ScrumTime in order to submit new features, approve cost estimates per feature, and participate in the product discussions that require their feedback. Using the terms "feature" and "release" make this sort of customer self-service more feasible and attainable.
In summary, I plan to add a new level into the existing ScrumTime work item hierarchy known as the feature level. This will provide a bridge between the customer point-of-view and the software engineer point-of-view. The benefit is that the tool will be able to track more of the customer feedback loop that is vital to successful software development projects.

Thursday, April 5, 2012

Micro ORM - Which to select?

Over the past week, I have been evaluating the three different micro orm choices in the ADO.NET arena. The three most popular seem to be Dapper, Massive, and PetaPoco. 

Scott Hanselman had an enlightening post on this topic about a year ago at: I believe that was just before PetaPoco started to gain notoriety as he covered only Dapper and Massive.

Massive is very flexible in that I am able to map the columns of the database to a dynamic object. However, I want to have strongly-typed objects in ScrumTime so this eliminates Massive.

Dapper appears to run the fastest from all that I have read and experienced on my own. It is very good at mapping database tables to strongly-typed objects for querying data. However, the process of inserting, updating, and deleting objects is not so clear. I have found that there is an extension to Dapper within the github Dapper repository that has insert, update, and delete capabilities. I am going to research that as soon as possible.

UPDATE (01/22/2013): Dapper has two separate extensions that offer two different excellent solutions to my concerns.  The two project names are Dapper.Contrib and Dapper.Rainbow.  The Dapper.Rainbow appears to be a perfect fit for what I was seeking.

PetaPoco is very promising. In fact, I have already made significant headway in using this ORM in my Product repository. I did have a small learning curve to overcome in using the SQL Builder that was included.

In general, I have debated the use of model attributes that identify the table name, primary key, etc. It seems to pollute the model with information that is not relevant to the model itself. It is more relevant to the method of storage which I intend to be encapsulated within the repository layer. But, I suppose it is much better than defining XML files as is the option in other more bloated ORMs. So, I am probably going to live with it because I cannot come up with an alternative that is as easy to follow as attributes. Therefore, I have no concrete concerns other than it seems to be a 'smell'.

At this time, I have reduced the three choices down to two choices: Dapper or PetaPoco. I suppose that is progress.

UPDATE (01/22/2013):  I found that Dapper and PetaPoco support of concept of "Multiple Querying" in order to solve one-to-many and many-to-one relationships.  I am a huge fan of this approach.  At this time, I am using PetaPoco only because it comes with a TT file for generating my model objects quickly.  I am sure I could create a TT for Dapper, but I have not had the time.