iLude

Returning every document of a Type from RavenDB

Running with Scissors

I have recently been working on a new idea and made the choice to use RavenDB for my persistent storage. For the most part its working out well. But I’ve run into two issues. The first I’ll hopefully deal with in a later post but the other was a real pain in my butt. Raven limits the returned query result sets to 128 documents regardless of how many documents match the query. Now Ayende claims that this is a good thing. I disagree and I’m not the only one. Yes you can override it….somehow, but knowing that this behavior exists and how to change it would need to be documented. I couldn’t find it after looking for a few hours and I’m more interesting in writing my software not fixing someone else’s.  I hope that Ayende and his team will address the documentation issue soon.

In my case I want to do some batch processing and need all the documents of a type and I don’t want to jump through flaming hoops to get them.  So here’s the extension method I came up with to solve my issue.

public static class RavenExtensions {

    public static IEnumerable<T> LoadAll<T>(this IDocumentSession session) {

      var count = 0;

      IDocumentQuery<T> results;

      do {

        var ofTypeT = string.Format("Tag:[[{0}]]", ReflectionUtil.GetFullNameWithoutVersionInformation(typeof(T)));

        results = session.LuceneQuery<T>().Where(ofTypeT).Skip(count).WaitForNonStaleResults();

        foreach (var result in results) {

          count++;

          yield return result;

        }

      } while (results.QueryResult.TotalResults > count);

    }

  }

This gives us a new session method that we return all of the documents when enumerated, but still obeys the small result set constraint enforced by the server.

I’ll wrap this up by saying that this method is for very specific uses. Ayende is correct that there are many problems that are the result of unbounded result sets. But I do not agree that the client API should make it more difficult that needed to do a common operation. I’ll leave you with this final link to a post that makes the same point.