Reporting In NoSQL

Rob Conery, creator of SubSonic, states his opinion about reporting in NoSQL database: Use the right tool for the right job. Consider splitting up your data in ‘domain-data’ and ‘reporting-data’. The domain-data is kept databases which fits best, like a document-store, object-database etc. The data for the reporting stored separately. So it’s a separation of data concerns.

First you may think: “Oh no, more work”. Two data-stores, that’s even more code, more libraries etc you have to maintain. But I’m not if that is really true. When I’m thinking about all the ‘pain’ you go through to map you domain into a normalized relational model. Especially when the relational-model is used for both, reporting and the application-domain.

For example I’ve once build a quite sophisticated survey application. Well I first designed my domain-model for representing a survey. Like domain-objects for a question, possible-answers, rating-criteria’s etc. I’ve build a nice model an then mapped it onto a relational representation. Now of course a survey-application also needs detailed reports. Well first the domain-model was easy to work with. But it wasn’t optimal for reporting. Things got worse because I tried to ‘hammer’ the reporting-’nails’ with a ORM-mapper. And it was a nightmare, because the ORM couldn’t handle complex queries appropriately . So I needed to tweak the domain-model to make it more reporting friendly. In summary, I spend a lot of time tweaking and changing the domain-model and ended with something, which was neither good for reporting nor a good domain-model.

When I think back, I could have saved a lot of trouble by separating the both parts completely. Use nice domain-model for the survey, preferably stored in object-database or maybe a document-database. This is data which is actively changed and updated. And then for reporting, additional data is stored In big, flat and demoralized tables.

Conclusion: I can imagine that separating data for different purposes can be a real benefit.

, ,

1 Comment

What Am I Working On?

I’m currently extending the dSail System-software.  The idea is to provide real-time data about the air flow on a sailing boat. This information can help you tweaking the sail position to reach the highest speed possible. In order to get this data sensor-nodes are attached to the sail. Each node has multiple sensors, like pressure, wind-speed etc. Each node send its data to an analysis software. I’m working on that analysis software. It parses the raw data, logs it and shows nice real-time charts.

Overview

Right now we’re adoption the software for other projects and purposes. The goal is to create a toolkit for parsing, storing and visualizing all kinds of sensor-data. Air-pressure, temperature, CO2-content, rotation and acceleration and so on and so forth. The sky it the limit.

Anyway, I really enjoy working on it =)

, ,

No Comments

Throwing Checked Exceptions Like Unchecked Exceptions in Java

 

Fooling Checked=

Fooling Checked Exceptions

Everyone who’s programming Java knows checked exceptions. You only can throw a checked exception when you’ve declare it. And you have to catch an checked exception, no matter whether it actually can occur.

First my opinion: I think check exception are a failure. The intention was good, but in practice they cause pain, boilerplate-code and in general cause more problems than they solve.

So in my code I avoid checked exceptions. Unfortunately the existing JAVA-API’s are littered with checked exceptions. So I still need to deal with them. The usual strategy looks like this:

try{
     someObject.doOperation();
} catch(OperationException e){
    throw new RuntimeException("See inner exception",e);
}

This works fine, but fills up the stack-trace with unnecessary garbage. (grey= garbage, black= important). And because you normally have multiple such rethrow-blocks in your code, the cause is nested somewhere in a huge stack trace:

Exception in thread "main" java.lang.RuntimeException: See inner exception
    at ch.gamlor.test.ExceptionBlogPost.methodWithoutCheckedException(ExceptionBlogPost.java:27)
    at ch.gamlor.test.ExceptionBlogPost.openSimulation(ExceptionBlogPost.java:20)
    at ch.gamlor.test.ExceptionBlogPost.demostrateStackTrace(ExceptionBlogPost.java:15)
    at ch.gamlor.test.ExceptionBlogPost.main(ExceptionBlogPost.java:11)
Caused by: ch.gamlor.test.OperationFailedException: Could not find appropriate example
    at ch.gamlor.test.ExceptionBlogPost.legacyMethodWithCheckedExceptions(ExceptionBlogPost.java:32)
    at ch.gamlor.test.ExceptionBlogPost.methodWithoutCheckedException(ExceptionBlogPost.java:25)
    ... 3 more

The Hack

Now recently I’ve discovered this post on StackOverflow, about unknown ‘features’ of the Java language. There someone showed how to throw checked exception without declaring them. Basically he combines the exception with generics and misuses the type erasure of the compiler, so that the type of the exception is erased =). Btw. the post includes another surprise: You can declare a class in methods =).

Here’s the code and more details how it works:

public final class UncheckedThrow {
    private UncheckedThrow(){}

    public static void throwUnchecked(final Exception ex){
        // Now we use the 'generic' method. Normally the type T is inferred
        // from the parameters. However you can specify the type also explicit!
        // Now we du just that! We use the RuntimeException as type!
        // That means the throwsUnchecked throws an unchecked exception!
        // Since the types are erased, no type-information is there to prevent this!
        UncheckedThrow.<RuntimeException>throwsUnchecked(ex);
    }

    /**
     * Remember, Generics are erased in Java. So this basically throws an Exception. The real
     * Type of T is lost during the compilation
     */
    public static <T extends Exception> void throwsUnchecked(Exception toThrow) throws T{
        // Since the type is erased, this cast actually does nothing!!!
        // we can throw any exception
        throw (T) toThrow;
    }
}

Now you just can use this method to ‘rethrow’.:

try{
    legacyMethodWithCheckedExceptions();
} catch (OperationFailedException e){
    UncheckedThrow.throwUnchecked(e);
}

And surprisingly this works! And even better, no rethrow-garbage, and also the throw location is right!

Exception in thread "main" ch.gamlor.test.OperationFailedException: Could not find appropriate example ;)
    at ch.gamlor.test.ExceptionBlogPost.legacyMethodWithCheckedExceptions(ExceptionBlogPost.java:32)
    at ch.gamlor.test.ExceptionBlogPost.methodWithoutCheckedException(ExceptionBlogPost.java:25)
    at ch.gamlor.test.ExceptionBlogPost.openSimulation(ExceptionBlogPost.java:20)
    at ch.gamlor.test.ExceptionBlogPost.demostrateStackTrace(ExceptionBlogPost.java:15)
    at ch.gamlor.test.ExceptionBlogPost.main(ExceptionBlogPost.java:11)

 

Fine Tuning The Hack

Now we’re facing the next problem. Supposed we’ve a method which returns a value and we use out little hack. Then we’ve a problem. Since the compiler doesn’t see the ‘exception’, it requires you to return something or throw something. Well that’s easy to fix. Change the signature from void to RuntimeException, so that you can throw it. But the implementation never actually returns, but throws the suppress checked exception.

public final class UncheckedThrow {
    private UncheckedThrow(){}

    // Now this returns an exception, so that you can satisfy the compiler by throwing it.
    // But in reality we throw the given exception!
    public static RuntimeException throwUnchecked(final Exception ex){
        // Now we use the 'generic' method. Normally the type T is inferred
        // from the parameters. However you can specify the type also explicit!
        // Now we du just that! We use the RuntimeException as type!
        // That means the throwsUnchecked throws an unchecked exception!
        // Since the types are erased, no type-information is there to prevent this!
        UncheckedThrow.<RuntimeException>throwsUnchecked(ex);

        // This is here is only to satisfy the compiler. It's actually unreachable code!
        throw new AssertionError("This code should be unreachable. Something went terrible wrong here!");
    }

    /**
     * Remember, Generics are erased in Java. So this basically throws an Exception. The real
     * Type of T is lost during the compilation
     */
    public static <T extends Exception> void throwsUnchecked(Exception toThrow) throws T{
        // Since the type is erased, this cast actually does nothing!!!
        // we can throw any exception
        throw (T) toThrow;
    }
}

Now you can use it anywhere!

private int methodWithoutCheckedException() {
    try{
        return legacyMethodWithCheckedExceptions();
    } catch (OperationFailedException e){
        throw UncheckedThrow.throwUnchecked(e);
    }
}

It’s not only limited to rethrows, you can throw a checked exception anywhere. However I think that’s a very bad idea!

private String demoJustThrow(){
    throw UncheckedThrow.throwUnchecked(new IOException("See, I'm not declared"));
}

Conclusion

I’ve demonstrated how to throw checked exception like unchecked ones. Yes, this is certainly a real evil hack. However I also think that avoiding unnecessary garbage in stack traces is a real benefit. So I’ve started to use this for rethrowing checked exceptions in my projects.

I’ve also to mention that this could be a weakness in the javac-compiler. I haven’t checked this against the language definition =).

The source-file is here. Try it yourself: UncheckedThrow.java

,

No Comments

The Way of the Whiteboard, Modern Hardware, Writing DSLs with LINQ and Component Relationships

, , , ,

No Comments

db4o: Tools For Adhoc Querying And Modification

Remember my very first post about db4o? There I’ve made the statement that the tool-support is very bad. Well I don’t revise my opinion here. The situation is still bad compared to the relational database world. Mostly that’s because relational databases have a giant user base, are well known and have some common interfaces (JDBC, ODBC, etc).

(All posts of this series: the basics, activation, object-identity, transactions, persistent classes, single container concurrency, Queries in Java, C# 2.0, client-server concurrency, transparent persistence, adhoc query tools)

Now enough complaining. Only complaining doesn’t help anyway!  In this post I’ll introduce two tools and show briefly, what you can do with them. Let’s start:

ObjectManager Enterprise (OME)

This is a tool which comes bundled with db4o. In the Java-Version it’s a Eclipse-plugin. In the .NET-version it’s a Visual Studio add-on.

For the .NET-version there are installers in the ‘ome2005’ and ‘ome2008’-directory of the db4o-distribution. Pick the one which corresponds to your Visual Studio version and install it. After restarting Visual Studio, you’ve a new buttons in you’re toolbar and a new menu-point ‘Tools’->’ObjectManager Enterprise’.

For the Java-Version there’s a zip-file in the ‘ome’-directory of the db4o-distribution. Unzip it somewhere. Open Eclipse, go to ‘Help’->’Install New Software’. Click on ‘Add’, then ‘Local’ and choose the directory you’ve unzipped the zip-file before. The click just through all the confirm-dialogs etc. Then restart Eclipse. Now there’s a new menu-point ‘OME’ in Eclipse.

Now you can connect to a db4o-database. Either you can connect to a running db4o-server or open a db4o-database-file directly.

Queries with OME

Now in the OME you don’t use a query-language for querying the database. This is quite a surprise. So how do you build queries then? You build then by drag and drop! On the left the OME shows a list of all known classes. When you expand such a class, you see the fields. Fields which are no primitives can be extended further. To build a query you choose a object-type, lets say ‘Person’. Then you expand the ‘Person’-class on the left, choose the fields you want to query and drag them into the ‘Query-Builder’-window. After sticking the criterions together, you can run the query. Furthermore you also can update field of persisted data.

Here a little screen-cast, which shows this:

To get an overview of the OME-Features, you also might take a look at this screen-cast.

LINQPad

Now I’m not a fan of this visual query-building. I prefer a query-language. But db4o doesn’t provide a language like SQL, but rather utilized the language of the platform. Now .NET brings a wonderful query-language, LINQ. And together with the excellent LINQPad you can use it like a traditional SQL-console. However some preparation is required:

  1. We need to add the db4o-assemblies. LINQPad has no idea about db4o, therefore we need to include the db4o database engine. To achieve this, go to the ‘Query’->’Advanced Query Properties’. On the dialog, click on ‘Add…’, then on ‘Browse…’. Browse to the db4o-assemblies. Add the ‘Db4objects.Db4o.dll’, ‘Db4objects.Db4o.Linq.dll’, ‘Cecil.FlowAnalysis.dll’ and the ‘Mono.Cecil.dll’-assemblies. Keep the ‘Advanced Query Properties’-dialog open for the next step.
  2. In the ‘Advanced Query Properties’-dialog open again the ‘Ad…’-dialog. Add you’re assembly which contains the stored classes. We need the classes in order to write LINQ-Queries.
  3. In the ‘Advanced Query Properties’-dialog go the tab ‘Additional Namespace Imports’. Add there the db4o-namespaces:
    Db4objects.Db4o.Linq
    Db4objects.Db4o
    Additional add the namespace of your stored classes, for example:
    MyProject.DomainModel
  4. Click on ‘Set as default for new queries’ so that you don’t need to repeat the three steps above next time. Then we’re done.

Queries with LINQPad

Now LINQPad is ready for db4o. Choose “C# statements” or “VB statement” in the language-chooser. Then you can write you’re query like this:

using(var db = Db4oEmbedded.OpenFile(@"C:\file-path\database-file.db4o")){
    var result = // query!!!
    result.Dump();
}

For example:

using(var db = Db4oEmbedded.OpenFile(@"C:\Users\Gamlor\Desktop\test.db4o")){
    var result = from Person p in db
                    where p.FirstName.Contains("Roman")
                    select new {p.FirstName, p.LivesAt.Street, p.LivesAt.City.Name};
    result.Dump();
}

As you see, you just use normal C# code. So you can run any LINQ-Query on you’re database.

Update with LINQPad

Well updating is also possible. LINQPad is nothing else than a C#-Console. So you just get the objects, change then and store the changes. For example:

using(var db = Db4oEmbedded.OpenFile(@"C:\Users\Gamlor\Desktop\test.db4o")){
    var update = (from Person p in db
                    where p.FirstName.Contains("Roman")
                    select p).First();
    update.FirstName = "Romanovski";
    db.Store(update);
    update.Dump();
}

Some LINQPad-Tips

Since LINQPad doesn’t know anything about db4o, but is only a interactive C#-console, you are responsible for db4o specific behavior. For example the activation-depth, update-depth etc also apply within LINQPad. So for example when all properties of a nested object are null, it is might not activated yet.

You might also write a little helper class library which you can use within LINQPad together with db4o.

Since LINQPad is a popular tool, you certainly find more resources on the internet.

Conclusion

I’ve introduced two tools for ad-hoc querying and modification of the db4o-database. The Object Manager with it’s graphical query builder and LINQPad. Both are useful for inspecting the database.

However I would like to see more powerful tools. When you know one, please tell me.

,

No Comments

Autofac 2.x

A new version of favorite dependency-injection-container Autofac is making progress. It follows the same principals, brings some new features etc. However they also refactored the API, so it isn’t compatible with the older versions.
This means, that migrating to the new version is some work. However there are useful new features.

Lots of them I’ve implemented on top of the old Autofac-version:

  • Collection Support ("Resolve All"): I’ve implemented something like this. Basically it allows you to resolve multiple implementation of a service. Very useful for some simple plugin-mechanisms. (not every application needs MEF)
  • Auto-Generated Factories: Allows you to create multiple instances of an service. The container is responsible for implementing a factory. This feature I’ve also implemented.
  • Owned Instances: Autofac has extremely good support for managing resources. Now you can inject a Owned<T> instance, which creates a new disposable scope. When you call Owed<T>.Dispose(), the service T, with its resources and dependencies are disposed.
    I’ve implemented something similar: All services exposing the IDisposable-interface create a own scope. As soon as you dispose the service the scope with its resources etc is disposed.

Autofac 2.1 already bring support for the .NET 4.0 stuff like Lazy<T> etc.

I keep an eye on Nicholas Blumhardt’s blog and as soon as Autofac is out of beta, I’ll migrate.

, ,

No Comments

Coupling

CouplingAfter the breakup with Jane Steve falls in love with Susan. This also shakes up the relations between the close friends of Steve and Susan. Anyway the six friends go through turbulent times together.

Coupling (IMDb, Wikipedia) is a hilarious sitcom about relations-ships,  sex and bottoms. Each episode follows the group of the six main characters. Often the episode isn’t a linear timeline, but a interleaving of the current events and previous ones. This interleaving is cleverly used to set up different context for each group. Mixing this different contexts leads then to damned funny misunderstandings. Anyway, the show does a great job playing with misinterpretation and expectations and has a great sense of humor.  
The choice of characters is optimal for such a show. There are two characters  (Steve and Susan) which are sensible and somewhat realistic, so that you can sympathize with them. The other four characters are just crazy fools and there you entertain the audience.

Coupling ran for four successful season. There’s also a U.S. and an Greek remake, so ensure that you pick the original.

Star-O-Meter: starstarstarstarstar (5/5)

Clip 1: Angry Susan

 

Clip 2: Jeff Kissed Someone

 

Clip 3: (same scene from two different perspectives)

, , , ,

1 Comment

Trailers of Doctor Who, Being Human and Survivors

Ok, lots of trailers of some of my favorite shows. It going to be a wonderful tv-year.

Doctor Who Series 5

First, my favorite sci-fi series Doctor Who. (blog post about season 1, 2, 3, 4, 4.5).  Trailer for series 5:

 

Being Human Season 2

Then my favorite super-natural comedy-drama (blog post): Trailer for season 2: 

 

Even a clip from the first episode…I could resist despite the spoiler-danger…=(

 

Survivor Season 2

And to end this post, a more serious drama: Survivors (blog post). Trailer for Season 2

, ,

2 Comments

Maven-Reference, Validation and Geek-Videos

, , , , ,

No Comments

The Unusuals

- Maybe you should sit down first…The Unusuals
- No, no, I don’t want to sit down, just tell me.
- Yeah?
- Yeah!
- Ok, you’re dead.
- That’s not funny.
- No, no, I’m not kidding, you’re legally dead…

Casey Shraeger is new to the homicide unit and finds herself in a unusual group of people. For example Leo Banks, who beliefs that he dies at age 42. Eric Delahoy who has a brain-tumor but is afraid to accept treatment. Another detective thinks he’s the best and doesn’t need a partner at all. So Shraeger tries to do her best fighting crime and fitting into the group.

The Unusuals (Wikipedia,IMDb) is a comedy-drama which follows a whole group of detectives. Each episode follows different cases and the relations-ships within the squad. Additionally a parts of the story is running over multiple episodes. So it’s not the usual dead body, 40-minutes investigation, hurray here’s the killer.
Honestly, after the first episode I nearly quit watching. However I kept watching and after few episodes I started to like it. Mostly because fell in love with the characters. The show doesn’t have deep an complex character-development. But the characters are just on the sweet spot between comic-like and still real.

End the end, the show is a entertaining mix crime-solving, funny moments and a pinch of drama. Unfortunately the won’t be a second season =(.

Star-O-Meter: starstarstarstarempty-star (4/5)

Trailer 1:

Trailer 2:

, , , , ,

No Comments