Sunday, 8 January 2012

2011 Conceptual Web Framework Utopia – Part 2, Testing


This short series of posts represents a snapshot of my thinking at the end of 2011, in terms of how I would ideally produce web applications using the .NET platform. In this, part 2, I’m going to cover my current testing strategy and preferences.
For this post, the plan is:
1.       General  Testing
2.       Endpoint Testing, using the Endpoint Testing Pattern
3.       Database Testing with RavenDb
I won’t be talking about higher-level tests or BDD that drive the TDD. The only variable there is the frameworks, which are generally interchangeable.

General Testing

My unit tests need to be fast, and they need to be readable – no interacting with databases then. See the last section for that.
To make unit tests readable I treat them as high-importance code. This doesn’t mean I take any longer, it means I have three simple rules that I nearly always use:
1.       Keep them short (3 lines is optimal, 7 means I don’t get to eat for the day)
2.       Hide assertions inside extension methods that read well (just like the GOOS boys)
3.       Wrap setup code in methods too
[Test]
public void Get_GivenIdForExistingBook_ShouldReturnViewModel_WithBooksDetails()
{
    var book = GetRandomBookSimulatedToExist();
   
    var viewModel = endpoint.Get(new UpdateBookLinkModel {Id = book.Id});
   
    viewModel.ShouldHaveDetailsFor(book);
}

Above is an example of what I consider to be an ideal test.
Context Specifications
With the above preferences applied, most of the time there are just 3 - 5 line test methods that follow like a given, when then. Sometimes, though, it really makes sense to use a context specification.
[SetUp]
public void WhenRequestingABookReview_AndTheSystemContainsBooks_WithDifferentGenres()
{
    var retriever = MockRepository.GenerateMock<IBookRetriever>();
    var endpoint = new ViewEndpoint(retriever);
   
    book = BookTestingHelper.GetBook(rating: 4);
    retriever.Stub(r => r.GetById(book.Id)).Return(book);
    
    SetupBookForSameGenreIncludingReviewBook();
    retriever.Stub(r => r.GetReviewedBooks(book.Genre.Id)).Return(allBooksForSameGenre);
    viewModel = endpoint.Get(new ViewBookLinkModel { Id = book.Id });
}
 
[Test]
public void ViewModelShouldHaveBooksTitle()
{
    Assert.AreEqual(book.Title, viewModel.Title);
}
 
[Test]
public void ViewModelShouldHaveBooksGenreName()
{
    Assert.AreEqual(book.Genre.Name, viewModel.GenreName);
}
 
[Test]
public void ViewModelShouldHaveBooksRating()
{
    Assert.AreEqual(book.Rating, viewModel.Rating);
}

If you look at the full source code for this class you’ll see there are another 6 tests below it. I think in this case it was the most readable way to test.
When Do They Run?
Always - these are the general tests that form part of the main suite for giving immediate feedback about any regressions.

Endpoint Testing

In part 1 my diagram mentioned “Endpoint-specific models”. This includes link (usually called request), view, and input models. By having them specific to each endpoint, the models are tested as part of the endpoint tests and not alone. I refer to this mentallly as the endpoint testing pattern, but it is not an official term.
For example:
[Test]
public void Get_GivenIdForExistingBook_ShouldReturnViewModel_WithBooksDetails()
{
    var book = GetRandomBookSimulatedToExist();
   
    var viewModel = endpoint.Get(new UpdateBookLinkModel {Id = book.Id});
   
    viewModel.ShouldHaveDetailsFor(book);
}

This test covers the endpoint and the models. There would be no additional tests for the models – they only exist to serve one action method on this endpoint so don’t need it.
Update Scenario
[Test]
public void Post_GivenUpdateModel_ShouldCreateDtoAndPassToBookUpdater()
{
    var model = new UpdateBookInputModel
              {
                 Genre       = "genres/9",
                 BookStatus  = BookStatus.Reviewed,
                 Title       = "Updated title",
                 Id          = "books/444",
                 Rating      =  3,
                 Description_BigText = "Updated description",
              };
 
    endpoint.Post(model);
    
    updater.ShouldHaveBeenCalledWithDtoMatching(model);
}

For these scenarios, a simple assertion is made that a service was called. Anything more involved than that is probably logic that doesn’t live there in an ideal world.
When Do They Run?
These will be ultra-fast tests that are part of the main suite of tests. Every time there is a change in code, these get run to check for regressions.

Database Testing With RavenDb

I’ve specifically name-dropped RavenDb because this is my utopia. In this world, we eat, sleep and breathe RavenDb for data access because it makes life a lot easier. You get all the benefits of NHibernate’s session, transactions & POCOs, without any of the pain of the SQL mapping or performance disasters.
Another benefit of RavenDb is the testing support. In fact, this is fantastic. A mini Ayende sits their inside your pc spinning up an instance of his document database in your computer’s memory. As far as I’m aware, this works exactly the same as a normal instance of the database. None of those SQL Lite – SQL Server mismatches.
Here’s a data access test:
[Test]
public void GetWishlistBooks_ShouldOnlyReturnBooks_OnTheWishlist()
{
    PopulateSessionWithBooksOfDifferentStatus();
    Session.SaveChanges();
    var fromSession = Session.Query<Book>().Where(b => b.Status == BookStatus.Wishlist);
    var fromRetriever = retriever.GetWishlistBooks();
    
    Assert.AreEqual(fromSession, fromRetriever);
}

Here is how the session is setup for testing….
public abstract class RavenTestsBase
{
    protected IDocumentSession Session;
    private EmbeddableDocumentStore store;
 
    [SetUp]
    public void SetUp()
    {
        store = new EmbeddableDocumentStore
                  {
                      Configuration = {RunInMemory = true,}
                  };
 
         store.Initialize();
 
         Session = store.OpenSession();
    }
 
    [TearDown]
    public void TearDown()
    {
        Session.Dispose();
    }
}

…..I agree - someone is taking the piss. This is so simple I’m worried the cleaner will take my job.
So You Run This With Your Main Test Suite?
They’re fast, but they’re no Ussain Bolt on happy pills….. I run these when any data access code changes and before important commits.

Conclusion

I hope you enjoyed reading about my testing preferences and conventions. Whether you agree with them or not, it’s important to tell yourself how you think the best way to do something is. This gives you a point of reference to take on and incorporate new information – it helps you to improve… and socialise with other developers in the pub.
The most important part is having conventions about how you test. This lets you create consistent tests, that require less cognitive resource to create, maintain and read. You can then put more focus on delivering software and understanding the problem domain.
In part 3 I’ll show you some of my other preferences for web frameworks. Most of them have been learned by playing with FubuMVC.

 

 

Sunday, 1 January 2012

DDD - Notes from the big blue book. Part 8

Chapter 15 - Distillation

  • Identify the core domain and make it small
  • Identity of the core domain should evolve through iterations
  • Create a short domain vision statement and revise it as you gain new insights
  • A domain vision statement gives the team shared direction
  • Different forms of highlighted core are useful for easily representing the important parts of the core domain eg. distillation document, flagged core
  • Make it effortless for a developer to know what is in or out of the core
  • Domain vision statements work in synergy e.g. with a segregated core

Chapter 16 - Large Scale Structures
  • Large systems that have well grouped modules and bounded contexts can still become unintelligible when there are many of them
  • Be prepared to drop a system metaphor if it gives poor guidance
  • Combine layered architecture with responsibility-driven design to achieve responsibility layers
  • Knowledge models may be appropriate when the roles and responsibilities in a domain can change at runtime
  • Pluggable component frameworks don't easily allow constant refactoring of the core domain
  • Large scale structures must enter the ubiquitous language

Chapter 17 - Bringing the Strategy Together
  • The three basic principles of strategic design are complementary
    • Context
    • Distillation
    • Large-scale structure
  • When tackling strategic design:
    • Draw a context map
    • Verify ubiquitous language is correctly in use
    • Verify there is a core domain and vision statement
    • Does the technology choice work for model driven design?
    • Does dev team have the skills?
    • Does dev team have knowledge and interest in the domain
  • Design decisions must reach the entire team

Sunday, 18 December 2011

2011 Conceptual Web Framework Utopia – Part 1, “Architecture”


Bad experiences scar us - forcing us to learn for our own survival. This evolutionary theory can also be applied to software. If you don’t learn from your mistakes, you’ll soon be on this guy’s plate:

So when creating web apps, what have my survival instincts really taught me?
They’ve taught me that almost nothing applies universally. Since I generally create larger-scale web apps, this article applies primarily to that context. Even in this context, though, trade-off benefits will vary for specific circumstances.
They’ve also taught me to isolate the domain, keep the web layer thin and listen to the wise words of people of like Uncle Bob, Udi Dahan & Eric Evans. They’ve even taught me (via Udi Dahan) to “justify why the simplest possible thing isn’t enough” – so expect lots of that too.
In this series of blog posts - reflecting my current thinking - I will be showing:
1.       The “Architecture” (clarify the quotes the later)
2.       My unit testing strategy
3.       A few other preferred patterns

What Does it Look Like?


First, when I talk about architecture, I apply it generally and mean any component of the system as a whole – not just the domain. This contradicts Uncle Bob and may be wrong, but I couldn’t think of another phrase that would be understood that encompassed domain and delivery mechanism(s).

One Model In One Model Out & Endpoint-specific Models

A recent addition I picked up is the OMIOMO (one model in one model out) convention from fubumvc – each action receives a request or input model and returns a view model (or some other model).
By making these models specific to each action, they are very simple data structures – keeping cognitive load to a minimum. It also saves you from having shared view models that introduce bugs when they change for one view and cause another to break.
Endpoint-specific models facilitate the Endpoint Testing Pattern – discussed in part 2.

Endpoints Talk to Domain Services in DTO Language

Transfer of information between endpoints/controllers and domain services is through simple DTO data structures.
Here’s an example:
Endpoint using a query service
public CreateBookViewModel Get(CreateBookLinkModel model)
{
    IEnumerable<GenreDto> genres = genreRetriever.GetAllOrderedByName();
   
    return new CreateBookViewModel(genres);
}

Implementation of query service
public IEnumerable<GenreDto> GetAllOrderedByName()
{
    var genres = session
                 .Query<Genre>()
                 .OrderBy(g => g.Name);
 
    return genres.Select(g => new GenreDto {Id = g.Id, Name = g.Name});
}
 
public bool CanFindGenreWith(string id)
{
    return session.Query<Genre>().Count(g => g.Id == id) > 0;
}

Why Bother with DTOs?

Critically, the use of DTOs protects the domain and your application – the contract between your application and domain is a lightweight data structure. With this loose contract, the domain model is free to evolve – and when you’re applying DDD, with the need to refactor often and sometimes aggressively – your domain model has no outside concerns to worry about.
When you do apply domain refactorings, as long as you still meet the contract – the DTO – your application remains unaffected.
I won’t explain the business value  of DDD.

Why Separate Services to Query and Command?

A philosophy in CQRS is to separate reads and writes for architectural flexibility. At the heart it is a simple pattern. Generally it appears to be used when reads and writes have a separate data model – but in my case, the domain data model is hidden – so it could be the same or different – we have the flexibility. But what about the cost……

Isn’t Hiding Your Data Access Bad?

In some cases – yes. But for my example, I’d argue – no….domain isolation is worth it. It is a very thin layer, two mouse clicks away and one interface agreement. Data access needs to know domain entities and the underlying data model – to gain the benefits of DDD we don’t want to expose that to the application layers.
I think a thin layer of domain service implementations (not interfaces) that take a dependency on the data access is ok. Some people put them in a separate project, some people chuck it all in the controller – I like this middle ground.
Data access is a tricky, context-specific beast that this is probably the least prescriptive of my ideals.
Using this pattern and RavenDb also gives rise to a good strategy for comprehensive testing of data access – you’ll see that in part 2 as well.

An Example From the Wild

You start out with a synchronous N-Tier application, but your domain model rapidly evolves – with the DTOs being passed between boundaries this is no more difficult than necessary because the domain is isolated.
Suddenly you have multiple bounded contexts, sharing aspects of what once was a single domain entity - so you want to switch to an asynchronous messaging based architecture. As long as you have a separate service for queries and commands, with your data model hidden, you can focus totally on the domain and its contract to the outside world – DTOs.
For those thinking this doesn’t happen – it is the exact situation I am in with the contract I am working on, and appears not uncommon from what I hear elsewhere. If the application followed these principles, changes would be isolated to the domain and we’d only have to worry about meeting service interface criteria – unfortunately there are swathes of inter-twined spaghetti code all through the application and domain that are affected – there’s your business value, son.

So…..

Cowboys have their opinion, those in ivory towers voice quite another. But experience and practice are crucial in finding out what really works and the contexts in which the arguments put forward by each are favourable.
Reflecting on experiences, using personal projects to test assumptions and try new ideas is a great way to get closer to that firm-footing of reality. If you keep doing the same – you’ll keep getting the same. I was pretty bored of those domain entity model binding issues and domain refactorings causing crippling breakages in my web layer - so here’s a small conceptual step, less destined for failure.
Also when you write posts like these, you force yourself to double-check blog posts and podcasts – just to make sure you correctly understood the topic. This reinforces what you’ve learnt, also saving you from any false assumptions you may be carrying on your sleeve.
Tune in for part 2.

Related Shizz

Wednesday, 14 December 2011

DDD - Notes from the big blue book. Part 7

Chapter 13 - Refactoring Toward Deeper Insight

  • Refactoring toward deeper insight may be a response to a problem or awkwardness in the design, where the root of the problem is sensed to be the domain model
  • May come even when the code looks tidy - but the language just doesn't fit the model
  • Domain refactorings involve multiple people and are extensively designed and deliberated over
  • Could be a refinement of ubiquitious language which is then expressed in the code
  • "If you wait until you can make a complete justification for a change - you've waited too long. Your project is already incurring heavy costs"
  • Refactor when:
    • Design does not express current understanding
    • Implicit concepts can be made explicit
    • Opportunity to make important parts of the domain more supple



Chapter 14 - Maintaining Model Integrity

  • Total unification of the domain model will not be feasible or cost-effective
  • When inside a context, ignore terminology from outside - focus on keeping that context pure
  • Important to understand the bounds of a context
  • Duplicate concepts = 2 implementations of the same concept
  • False cognates = 2 concepts under the same name
  • Create a visual diagram of all the bounded contexts and translations on a project
  • Names of bounded contexts should enter the ubiquitous language
  • Shared kernel, conformist, open host service, published language
  • Anticorruption layers can be used with external systems or legacy code
  • Hard for multiple teams to work on one bounded context
  • For an existing project, draw up the bounded contexts AS THEY ARE


Sunday, 4 December 2011

DDD - Notes from the big blue book. Part 6

Chapter 11 - Applying Analysis Patterns

  • Have to make calculated compromises to the design to allow for good implementation
  • Parts of the program we don't suspect can benefit from a domain model - they can be uncovered with analysis patters
    • Example in this chapter was the nightly script that was run
  • Analysis patterns are more guiding than prescriptive

Chapter 12 - Relating Design Patterns to the Model
  • Design patterns can be applied to domain models by thinking about them in a different way - from a domain logic perspective
  • When applying a design pattern in the domain, first check whether it is a good fit for the domain concept
  • Example used in this chapter was the composite pattern with nodes representing shipment routes

DDD - Notes from the big blue book. Part 5

Chapter 9

  • To find implicit concepts
    • be aware of hints in the design
    • search them out
    • listen to language used and learn
    • Scrutinise awkwardness of design
  • Scrutinise code that does awkward things and is hard to explain
  • Domain experts may be inconsistent - this may indicate an implicit concept
  • Make constraints explicit
  • Look at analysis models/patterns
  • Ideally do not want to make procedures a part of the domain model - unless they are spoken about

Chapter 10

  • State post conditions of operations and invariants of classes and aggregates - ** maybe as unit test? **
  • Models are awkward because they are missing pieces - successive refactorings lead to conceptual contours which clarify the model
  • Decompose design elements consistently with the domain
  • A requirement that forces extensive change is a sign that the model does not fit
  • ** Closure of operations **
  • Carve off sub-domains

Saturday, 19 November 2011

Fun with Fubu's Html Conventions


I recently blogged about being impressed with conventions in Fubu MVC that let you decide the rules that govern how actions are discovered, how routes are constructed and displayed, and how output/view selection occurs.
Whilst on my travels in fubu land, the next feature that left me in awe (of the fanboy variety [of awe]) were the html conventions. Read on to learn how they out-muscle and bully ASP.NET MVC ’s display/editor templates for all their dinner money.
As a quick caveat: I am not an expert – these are just my initial findings.
The following example comes from one of my current personal projects.

Establish Context () =>

My first opportunity to apply html conventions came on the create book page hidden in site administration. Take a look at the CreateBookInputModel’s public properties:
public class CreateBookInputModel
 {
     public String Title { getset; }

     public String Genre { getset; }

     public String Description_BigText { getset; }

     public BookStatus BookStatus { getset; }

     public IList<StringWrapper> Authors { getset; }

     public HttpPostedFileBase Image { getset; }
 }

This public interface would make an excellent contrived example, as it demonstrates a number of different opportunities which get progressively more interesting. But this is a naturally-occurring story.
My goal is to keep the corresponding Spark view very simple – here’s an idyllic vision minus the fluff:
<p>
    ${this.LabelFor(m => m.Title)} 
    ${this.InputFor(m => m.Title)}
</p>
<p>
    ${this.LabelFor(m => m.Description_BigText)}
    ${this.InputFor(m => m.Description_BigText).Attr("rows", "5")}
</p>
<p>
    ${this.LabelFor(m => m.BookStatus)}
    ${this.InputFor(m => m.BookStatus)}
</p>
<p>
    ${this.LabelFor(m => m.Genre)}
    ${this.InputFor(m => m.Genres)}
</p>
<p>
    ${this.LabelFor(m => m.Authors)}
    ${this.InputFor(m => m.Authors)}
</p>
<p>
     ${this.LabelFor(m => m.Image)}
     ${this.InputFor(m => m.Image)}
</p>

Let’s start at the Top and Work Down 

Strings

public String Title { getset; }
 
public String Genre { getset; }
Simple – strings work straight out of the box with a nicely-named html input as you would expect.

Special Strings

public String Description_BigText { getset; }

Question: When is a string not a string?
Answer: When I suffix the property name with “_BigText”.
So what is it now then? It’s…..a big blob of text that needs to be represented by a text area – not a text box. To make this happen, head over to my FubuRegistry.
First we need a label:
HtmlConvention(x =>
     x.Labels
      .If(e => e.Accessor.Name.EndsWith("_BigText"))
            .BuildBy(er => new HtmlTag("label").Text(er.Accessor.Name.Replace("_BigText""")        ))
);

Here, the variable x is the registry, which I’m adding a new label convention to. What I’m saying is, if the property name ends with “_BigText”, build the view portion by using a html label whose text will be the property name, minus the “_BigText” marker (e.g. “Description”).
Now we need an editor to go with the label:
HtmlConvention(x =>
    x.Editors
      .If(e => e.Accessor.Name.EndsWith("_BigText"))
                .BuildBy(er => new HtmlTag("textarea"))

Pretty simple, eh? I’ve just told it to return a text area when the property name ends with our special marker – it is smart enough to give the text area the correct name so you can post values.

Enums

Here’s a favourite, and something I dreamt of doing with ASP.NET MVC, but was not possible with editor templates.
public BookStatus BookStatus { getset; }

Dear gods of web programming, every time you see an enum to be posted back please be so kind and give me a drop down with each enum member.
HtmlConvention(x =>
     x.Editors
      .If(e => e.Accessor.PropertyType.IsEnum)
           .BuildBy(er =>
                   {
                      var tag = new HtmlTag("select");
                      var enumValues = Enum.GetValues(er.Accessor.PropertyType);
                      foreach (var enumValue in enumValues)
                      {
                         tag.Children.Add(new HtmlTag("option").Text(enumValue.ToString()));
                      }
                      return tag;
                    })

Once you work out which variables correspond to the property type (er.Accessor.PropertyType) it’s pretty simple to get all the values and make a drop down. Again, the tag you return will automatically have the property name applied, so you can bind the selected value to it nicely.

String Collections

Caveat: fubu mvc doesn’t support binding to a collection of strings yet or IEnumerables (you can add your own, but it’s a bit of work). I just use StringWrapper  – which has one string property, “Text”; and ILIst<T> which is supported for binding collections to.
public IList<StringWrapper> Authors { getset; }

Whenever I see a collection of strings I always want a text box and add button, down which lets you type in text items and build up a list to be posted back. By using some class-name conventions, you can write the JQuery once and it works wherever you use the class name.
But by using fubu-mvc conventions – I write the JQuery once and the class once – the rest is taken care of:
HtmlConvention(x =>
    x.Editors
     .If(e => e.Accessor.PropertyType.IsAssignableFrom(typeof (IList<StringWrapper>)))
               .BuildBy(er =>
                       {
                          var tag = new HtmlTag("div").AddClass("hasHiddenGroup");
                                                                                            
                          tag.Children.Add(
                                       new HtmlTag("input")
                                           .Attr("type""text")
                                           .Attr("name", er.Accessor.Name)
                                          );
                                                    
                          tag.Children.Add(
                                       new HtmlTag("a")
                                            .Attr("href""#")
                                            .Text("add")
                                            .AddClass("addItem")
                                           );
 
                          tag.Children.Add(new HtmlTag("ul"));
 
                          return tag;
                        }));

Start with a div – and give it the magic sauce – “hasHiddenGroup” class (don’t know why I called it that – blame the magic sauce) – then add a text box with the property’s name. Next to that text box, add a link called add.
To clarify, the conventions will build this:
<div class="hasHiddenGroup">
    <input type="text" name="Authors">
    <a href="#" class="addItem">add</a>
   <ul></ul>
</div>

..which means that dash of JQuery can be applied site-wide to wire up the add button - allowing that empty to list to be built up. It then renames each item in the list so they are posted back and can be used to model-bind to.
You can see the simple JQuery code here.

File Uploads

Here’s an easy one:
HtmlConvention(x =>
    x.Editors
      .If(e => e.Accessor.PropertyType.IsAssignableFrom(typeof(HttpPostedFileBase)))
                 .BuildBy(er => new HtmlTag("input").Attr("type""file"))
);

Closing Thoughts

No doubt about it, the Fubu MVC html conventions go a lot deeper, and are less than brittle than ASP.NET MVC’s Display/Editor templates. When applied to a real world project, the time-savings and bug-reduction have massive potential.
But that’s a production app - this example was based on a small toy project. What may happen in the wild is a mystery
·         What happens if the markup is complex and is not so easily created in code?
·         What happens if need to apply specific classes?
·         What about when I want to override conventions – what pitfalls await me?
I look forward to finding out the answer to these questions, and striving to maintain my intellectually-crippled views.
On a final note you can also use fubu mvc’s html conventions in ASP.NET MVC.