Wednesday, March 31, 2010

RIA and DTO

At the moment (RC bits) WCF RIA doesn’t support custom DTO as parameters for domain service operations. If you define

Code Snippet
[DataContract]
public class MyCustomDTO
{
    [DataMember]
    public int MyPayload { get; set; }
    [DataMember]
    public string MyOtherPayload { get; set; }
}

You won't be able to do this

Code Snippet
[Invoke]
public void MyOperation(MyCustomDTO data)

Compiler will greet you with this encouraging message:

Operation named 'MyOperation' does not conform to the required signature. Parameter types must be an entity type or one of the predefined serializable types.

Well, don’t be scared, there is workaround (as usual) to make our custom DTO “RIA friendly” (or Entity type in RIA terms).

You need to do 2 simple but not obvious things

1. Make your class identifiable by providing RIA with identity property(ies)

Code Snippet
public class MyCustomDTO
{
    [Key]
    public int MyDummyKey { get; set; }

    public int MyPayload { get; set; }
    public string MyOtherPayload { get; set; }
}

Note that we don’t need WCF data contract attributes anymore – RIA code generator takes care of it

2. Expose our “entity” via domain service dummy Query

Code Snippet
[Query]
public IQueryable<MyCustomDTO> GetMyCustomDTODummy() { return null; }

Now you can successfully compile your project and use your custom operation with custom DTO client side

Code Snippet
var ctx = (OrderContext)orderDomainDataSource.DomainContext;
var data = new MyCustomDTO
{
    MyPayload = 13,
    MyOtherPayload = "Test"
};
ctx.MyOperation(data);

but what about more complex DTOs with sub lists (data graphs)? well, just wait for the next post ;)

Friday, March 26, 2010

WCF RIA and concurrency

I have been asked recently on WCF RIA Services forum about handling concurrency in WCF RIA. Well, the problem is – there is no official information about how to do it ;) There was special method for handling concurrency on server side, but it was deleted in RIA RC bits

Removal of Resolve methods from DomainService

In the previous release, you could provide a method for customizing how concurrency conflicts are resolved. For the RC release, this feature and all related members have been removed. Providing a common method signature for resolving conflicts in different types of DALs could not be supported in this release.

which means we have only one option now – client side concurrency resolution. It’s not trivial but doable.

First of all – define how you will address concurrency server side (in your model). Standard approach is to use concurrency guard field. For MS SQL server it’s usually RowVersion : timestamp

image

then you have to ask your client – what concurrency resolution scenario do we need? Last win? Unlikely ;)

Normal approach is to notify user about concurrency conflict and allow to resolve it. Don’t make this process too complex though – users don’t like complexity 

Consider this code, which implements generic concurrency resolution handler for RIA

Code Snippet
/// <summary>
/// Submit behaviours for DomainContext:SubmitChanges
/// </summary>
public static class SubmitBehaviour
{
    public static void WithConcurrencyResolution<T>(SubmitOperation operation) where T : Entity
    {
        // concurrency conflict
        if (!operation.HasError || !operation.EntitiesInError.Any(c => c.EntityConflict != null)) return;
        
        var result = MessageBox.Show(
            "The entity was modified by other user, overwrite those changes?\r\n(Cancel will refresh your data to the most recent store state)",
            "Concurrency conflict", MessageBoxButton.OKCancel);

        var ctx = (DomainContext)operation.UserState;
        var query = (EntityQuery<T>) ctx.InvokeMethod("GetQuery", null);

        // query for changes only (or current page only)
        var changes = ctx.EntityContainer.GetChanges().Where(c => c.EntityState == EntityState.Modified).OfType<T>();
        var keys = typeof (T).GetProperties().Where(p => p.GetCustomAttributes(typeof (KeyAttribute), true).Count() != 0).ToList();
        query = query.Where(GetWhere(changes, keys));

        if (result == MessageBoxResult.OK)
        {
            ctx.Load(query, LoadBehavior.MergeIntoCurrent, (Action<LoadOperation>) (c => ctx.SubmitChanges(WithConcurrencyResolution<T>, ctx)), null);
        }
        else
        {
            ctx.Load(query, LoadBehavior.RefreshCurrent, null, true);
        }
        operation.MarkErrorAsHandled();
    }

    // make lambda {c => (c.Key1 == value1 && c.Key2 == value2) || (same for change 2)}
    private static Expression<Func<T, bool>> GetWhere<T>(IEnumerable<T> changes, IEnumerable<PropertyInfo> keys)
    {
        Expression currentGroup = null;
        Expression prevGroup = null;
        var param = Expression.Parameter(typeof(T), "c");
        foreach (var change in changes)
        {
            Expression prev = null;
            Expression current = null;
            // composite keys
            foreach (var key in keys)
            {
                var prop = Expression.Property(param, key);
                var value = Expression.Constant(change.GetPropertyValue(key.Name));
                current = Expression.Equal(prop, value);
                if (prev != null)
                {
                    current = Expression.AndAlso(prev, current);
                }
                prev = current;
            }
            currentGroup = current;
            if (prevGroup != null && currentGroup != null)
            {
                currentGroup = Expression.OrElse(prevGroup, currentGroup);
            }
            prevGroup = currentGroup;

        }
        return Expression.Lambda<Func<T, bool>>(currentGroup, new[] { param });
    }
}

The idea behind this voodoo (yes, each time we forced to create expression tree we curse a kitten) code is simple though

1. Submit change set, server side will try to apply it and if there is concurrency error it will return FIRST concurrency error to client

2. If we want to force our changes to be applied – reload changed data with LoadBehavior.MergeIntoCurrent – thus refreshing our RowVersions and keep changes we made, then resubmit until we receive clean response

3. If we want to refresh our data to latest DB state – just reload changed subset of records with LoadBehavior.RefreshCurrent – thus losing our changes

The usage is extremely simple

Code Snippet
private void SaveButtonClick(object sender, RoutedEventArgs e)
{
    orderDomainDataSource.DomainContext.SubmitChanges(SubmitBehaviour.WithConcurrencyResolution<Order>, orderDomainDataSource.DomainContext);
}

I deeply believe this approach is not ideal but at least it works. Now. Out of the box.

PS. RIA team has to consider better approach – client should provide server with resolution mode – server has to resolve accordingly

It is all about CRUD. (Part 2)

Ok, now we have base generic class that will do standard CRUD and GetQuery for us (see previous post). However real life impersonated as your current client is full of surprises and fancy requirements ;) What if we want to do custom CRUD or extend standard one? For example, consider this classic schema

image

What if we want to execute some custom business logic each time we add new Order and update OrderItem? Solution (as usual) is simple

Code Snippet
[EnableClientAccess]
public class OrderService : CompositeServiceBase<Order, TestEntities>
{
    public override void Insert(Order item)
    {
        base.Insert(item);
        // Put custom logic here for adding Order if required
        // this operation is inside transaction scope - throw exception to rollback changes
        
    }
    
    public void UpdateOrderItem(OrderItem item)
    {
        // Put custom logic here for updating OrderItem if required

    }

yes, just override corresponding CRUD operation for root type (Order) and add method with correct signature (RIA uses name conventions by default) for child entity (OrderItem). You might have noticed that comments saying something about transactions…let me explain what’s going on here – by default all CRUD operations for DomainService executed outside transaction scope (we just attach, insert, delete entities to EF’s context).

However imagine that we need to call stored procedure from our CRUD method as extension for standard operation and this SP will modify something (calculate your salary for example) – we need to make sure that this calculation will be atomic as well as standard CRUD (which is maintained by EF when RIA call ObjectContext.SaveChanges()) That’s why we need this base class

Code Snippet
/// <summary>
/// Provides base repository and transactional support + common entity processing before save
/// </summary>
/// <typeparam name="T">ObjectContext</typeparam>
public abstract class DomainServiceBase<T> : LinqToEntitiesDomainService<T>
    where T : ObjectContext, IObjectContext, new()
{
    public override void Initialize(DomainServiceContext context)
    {
        base.Initialize(context);
        DataContext.Current = ObjectContext;
    }

    public abstract void ProcessEntityBeforeSave(EntityObject entity);

    public override bool Submit(ChangeSet changeSet)
    {
        // do base CRUD inside transaction
        bool result;

        var changes = changeSet.ChangeSetEntries
            .Where(c => c.Operation == DomainOperation.Insert || c.Operation == DomainOperation.Update)
            .Select(c => c.Entity).Cast<EntityObject>();

        foreach (var entity in changes)
        {
            ProcessEntityBeforeSave(entity);
        }

        using (var tx = new TransactionScope(TransactionScopeOption.Required, new TransactionOptions { IsolationLevel = IsolationLevel.ReadCommitted }))
        {
            result = base.Submit(changeSet);
            if (!ChangeSet.HasError)
            {
                tx.Complete();
            }
        }

        return result;
    }

ProcessEntityBeforeSave method can be defined in derived class to implement common entities processing – in our example it can be assigning LastUpdated field to current DateTime

Code Snippet
public override void ProcessEntityBeforeSave(EntityObject entity)
{
    UpdateHelper.UpdateCommonFields(entity);
}

You might wonder what is DataContext.Current? well… wait for next post about Using Repository pattern in RIA ;)

Wednesday, March 24, 2010

It is all about CRUD.

I have been developing line of business (LOB) applications for more than 10 years by now, and I can tell that in vast majority of cases it is all about Search and basic data manipulations or simply CRUD (CRUD stands for Create, Update, Delete). Yes, sometimes you need to do funky stuff like workflows, but 90% is still CRUD, Search, CRUD, Search, CRUD and Search…
And in real life this CRUD is not simple one/two tables (favourite for demoing stuff) – one UI screen to do data manipulations. It is usually very complex object graphs (or composites) on the server with complex UI as a frontend. Lots of interrelated tabs, lists, sub lists, one-to-many, many-to-many and all the hell of relational modelling behind.
Let’s see how latest bits of WCF RIA (RC at the moment) address simple (and not so simple) CRUD operations over composites.
As WCF RIA developer you have to be aware of Compositional Hierarchies Simple speaking, you can define a "part-of" relationship inside you metadata by using [Composition] attribute. This will affect client-side generated entities behaviour (changes to leaf element will pop up changed flag thru object graph to root element) and server side (you can use ChangeSet.GetAssociatedChanges to retrieve associated changes). There is great post about it here
Ok, enough theory, let’s give it a go. Consider this object graph

clip_image001
Pretty simple, right? Just two levels of hierarchy, each element of List1 has two sub lists. (In my current project we have 3 levels of sub lists on average and up to 10 lists per hierarchy). Anyway, EF model is created, metadata is defined as

Code Snippet
[MetadataTypeAttribute(typeof(Metadata))]
public partial class Root
{
    internal sealed class Metadata
    {
        // Metadata classes are not meant to be instantiated.
        private Metadata() { }

        [Include, Composition]
        public EntityCollection<List1> List1;
    }
}

[MetadataTypeAttribute(typeof(Metadata))]
public partial class List1
{
    internal sealed class Metadata
    {
        // Metadata classes are not meant to be instantiated.
        private Metadata() {}

        [Include, Composition]
        public EntityCollection<List2> List2;

        [Include, Composition]
        public EntityCollection<List3> List3;
    }
}

Ok, next step is to create DomainServiceclip_image002

Great, almost 200 lines of code were generated for ComplexCompositeService class. We need to slightly tweak it to load the whole object graph though

Code Snippet
public IQueryable<Root> Get()
{
    return this.ObjectContext.Roots.Include("List1.List2").Include("List1.List3");
}

Now it’s time to quickly model simplest UI for basic CRUD. It looks like this

ui

Will it work? Of course NOT! ;) And the reason is – almost 200 lines of generated code. Have I told you I hate code generators and usual cr@p they produce? In this particular example generated code is

  • ugly formatted
  • repetitive
  • doesn’t work

what a wonderful combination ;) However let’s explore why it doesn’t work and how to fix it. Do simple test – add new element into List1 and press Save. Boom! Exception fires here

Code Snippet
public void UpdateRoot(Root currentRoot)
{
    this.ObjectContext.Roots.AttachAsModified(currentRoot, this.ChangeSet.GetOriginal(currentRoot));
}

unfortunately, this.ChangeSet.GetOriginal(currentRoot) is null for root element (it makes sense because it wasn’t physically modified, it was just marked as modified when we added new sub elements to composite) and AttachAsModified fails. Ok, it’s easy to fix

Code Snippet
public void InsertRoot(Root root)
{
    if ((root.EntityState != EntityState.Detached))
    {
        this.ObjectContext.ObjectStateManager.ChangeObjectState(root, EntityState.Added);
    }
    else
    {
        this.ObjectContext.Roots.AddObject(root);
    }
}

Great! Now we can save! Don’t be too excited though ;( Make test a bit harder – add two new lines to List1. Boom! Same place, different exception ;) Now it’s EF refuses to attach root element with two new sub items (we’ll discuss this amazing EF behaviour [I’d rather call it nasty bug] in later posts). Ok, I am tired of fighting these endless problems. The bottom line is – the code generated by RIA project template is useless for composites and complex CRUD.
But don’t run and hide! There is solution. And this solution is as simple as this

Code Snippet
[EnableClientAccess]
public class ComplexCompositeService : CompositeServiceBase<Root, TestEntities>
{
    // yes, it's empty!!! and you have got all CRUD + Search for free!
}

yes, zero lines of code is all you need! ;)
ok, let me explain how this magic works.

Code Snippet
/// <summary>
/// Provides base CRUD for type T (object graph root) + common Query method [Get] with whole object graph scope inclusion
/// Child sets of T have to be marked with [Include, Composition]
/// </summary>
/// <typeparam name="T">Root type</typeparam>
/// <typeparam name="T1">ObjectContext</typeparam>
public class CompositeServiceBase<T, T1> : DomainServiceBase<T1>  
    where T : EntityObject
    where T1 : ObjectContext, IObjectContext, new()

the basic idea is simple – generalise base CRUD operations and base querying based on composition scope defined by Include and Composition attributes.
First of all for composites you can expose only methods for your composite's root entity Update, Insert, Delete and using ChangeSet.GetAssociatedChanges apply the whole changeset to EF ObjectContext

Code Snippet
public virtual void Insert(T current)
{
    AttachAndApplyChanges(current);
}

public virtual void Update(T current)
{
    AttachAndApplyChanges(current);
}

public virtual void Delete(T current)
{
    AttachAndApplyChanges(current);
}

private void AttachAndApplyChanges<TElement>(TElement current) where TElement : EntityObject
{
    ObjectContext.ContextOptions.LazyLoadingEnabled = false;
    ObjectContext.AddObject(current); // attach the whole graph as added
    ApplyChangesToGraph(current);
}

// recursively traverse the changeset and apply it to ObjectContext
internal void ApplyChangesToGraph<TElement>(TElement current) where TElement : EntityObject
{
    var childSets = GetChildSets(current.GetType());
    var self = GetType().GetMethod("ApplyChangesToGraph", BindingFlags.Instance | BindingFlags.NonPublic);

    foreach (var set in childSets)
    {
        var expr = CreateLambda(current, set);
        
        foreach (var item in ChangeSet.GetAssociatedChanges(current, expr))
        {
            var apply = self.MakeGenericMethod(item.GetType());
            apply.Invoke(this, new[] { item });
        }
    }

    ApplyState(current);
}

private void ApplyState(EntityObject current)
{
    var state = EntityState.Unchanged;
    var operation = ChangeSet.GetChangeOperation(current);
    if (operation == ChangeOperation.Insert) return;
    if (operation == ChangeOperation.Update) state = EntityState.Modified;
    else if (operation == ChangeOperation.Delete) state = EntityState.Deleted;

    if (current.EntityState == EntityState.Detached) ObjectContext.AddObject(current); // deleted entities

    ObjectContext.ObjectStateManager.ChangeObjectState(current, state);
}

private static IEnumerable<PropertyInfo> GetChildSets(Type elementType)
{
    // explore buddy classes for metadata
    var metadataClassType = elementType.GetCustomAttributes(typeof(MetadataTypeAttribute), true).Cast<MetadataTypeAttribute>().First().MetadataClassType;
    var metadataChildSets = metadataClassType.GetMembers().Where(p => p.GetCustomAttributes(typeof(CompositionAttribute), true).Count() != 0);
    var childSets = elementType.GetProperties().Where(p => metadataChildSets.Select(m => m.Name).Contains(p.Name));
    return childSets;
}

private static Expression<Func<TElement, object>> CreateLambda<TElement>(TElement current, PropertyInfo set) where TElement : EntityObject
{
    // make lambda {c => c.CompositionPropertyName}
    var param = Expression.Parameter(current.GetType(), "c");
    var prop = Expression.Property(param, set);
    var expr = Expression.Lambda<Func<TElement, object>>(prop, new[] { param });
    return expr;
}

It might seems as a bit of voodoo code, but actually it’s quite simple – we just recursively traverse change set and apply proper entity state to EF unit of work

Secondly – common Get

Code Snippet
public virtual IQueryable<T> Get()
{
    // auto-include child sets based on [Composition] attribute
    BuildCompositionScope(typeof(T), string.Empty);
    var setProperty = ObjectContext.GetType().GetProperties().Single(pi => pi.PropertyType == typeof(ObjectSet<T>));
    var query = (ObjectQuery<T>)setProperty.GetValue(ObjectContext, null);
    foreach (var scope in _compositionScope) query = query.Include(scope);
    return query;
}

private readonly List<string> _compositionScope = new List<string>();

private void BuildCompositionScope(Type elementType, string scope)
{
    var childSets = GetChildSets(elementType);
    if (childSets.Count() == 0)
    {
        if (scope.Length != 0) _compositionScope.Add(scope.Remove(0, 1)); // add final path
        return;
    }
    foreach (var set in childSets)
    {
        // list vs prop
        var type = set.PropertyType.IsGenericType ? set.PropertyType.GetGenericArguments()[0] : set.PropertyType;
        BuildCompositionScope(type, scope + "." + set.Name);
    }
}

again – simple idea – instead of doing ObjectContext.Roots.Include("List1.List2").Include("List1.List3");
we build eager loading scope automatically based on composition scope definition metadata
And the best part is – it works. For any change set, any object graph. Out of the box.

…to be continued (how to extend base CRUD of CompositeServiceBase with your custom logic)

Sunday, March 21, 2010

Manifesto

I have been in software development since late 90s. I have seen many software project, many developers, many lines of code. And through all this I came up to my simple manifesto. Be proud of what you've done. Consider your code as a short story. Make it easy to read. Make intentions clear. Make storyline nice, short and clean. Write for others, not for yourself.