iDoc: Hedtek’s open source community documentation system

iDoc screenshot

iDoc is an open source documentation system for community use; to provide help pages and other documentation.

iDoc is more than just a plain old vanilla documentation system where users are presented with help pages written by experts.  We believe is important for the users of an application to be able to contribute to documentation and help others to learn to use an application. iDoc enables this in two ways:

  • Firstly, users can comment on help pages and elicit further information or suggest improvements.
  • Secondly, users can themselves create and edit help pages.

The latter ability is parameterisable, for more conservatively-set-up help systems iDoc’s default behaviour can be changed to limit the generation of pages to ‘experts only’.

iDoc is particularly media friendly in that it allows the addition of videos and images to help pages and to comments, and there is a variant iDoc that allows for embedded SPARQL queries to appear in the pages in various ways (link to be added here to the fishDelish iDoc). The latter will be important in educational activities around SPARQL.

Why another documentation system?

There were several motivations that inspired this project.

The foremost reason was that we wanted an easy-to-use documentation system for the Manchester PLE, where our desire was to host very short videos to help newcomers use the system. This isn’t to imply that the PLE is hard to use, but rather that some potential users will, eg, have no idea what a blog is, nor any idea of how to use a blog. Equally, we don’t see ourselves producing all the documentation, rather we expect some documentation to be produced and improved by our communities of users.

Most documentation systems we investigated involved the use of some specialised markup to produce content from some other source. This isn’t desirable for a system geared towards non-techie users of an application, and even if the suers were technically skilled, we imagine that they are unlikely to want to learn yet another syntax for creating content. It’s for this reason that iDoc provides a WYSIWYG editor to create pages and add comments. Users can even use the same editor to embed videos from YouTube and Vimeo in both help text and comments.

The second reason for iDoc’s creation was to test a new development approach for use within Hedtek. This approach is an agile user story-driven approach as part of a Behavior Driven Development (BDD) method. There are various advantages to BDD, but pretty much its a scenario-based design technique, which leverages the automated system development testing espoused by Test Driven Design (TDD) and, with practice, enables accurate estimations of the development resource needed to implement the system under consideration. Also importantly, BDD uses iterations in the development, in a similar way that SCRUM uses sprints to deliver working systems at the end of each sprint. We wanted a smaller (but non-trivial and useful) project to get up to speed and make sure we were comfortable in the process before using the process in larger projects. We are glad to say that BDD worked tremendously well for us and is now our default approach for all new projects we undertake.

Finally, there was the desire to have a non-trivial project developed in a carefully engineered way as an example in a new Software Engineering course we are developing for and delivering in the Advanced Master’s programme in the University of Manchester’s School of Computer Science. (Update early 2011: unfortunately we had for various reasons  to adopt a TDD in Java approach for this course, rather than a superior BDD in Ruby on Rails approach. This somewhat retrograde position is currently being reviewed.)

The current iDoc status

After two of four iterations, iDoc has reached the stage where it provides useful documentation facilities. It provides a clean URL syntax for pages, a WYSIWYG editor for content and comments, and drag-and-drop facilities for positioning pages in a documentation tree. Users can sign up, develop and publish page content, and comment on pages. There are some administration features that can, if so desired, limit who can create and edit help page body content, and there is the ability to select community moderators who can remove unsanitary comments. There is also search functionality help find relevant parts of the documentation; search facilities are capable of case-insensitive searches, wildcards, boolean queries and other niceties of advanced full-text search.


iDoc has been developed with the latest stable Ruby on Rails stack (version 2.3.5). It requires backing by a database, and was developed using a PostgreSQL database. However, iDoc is currently relational-database agnostic. The site uses Haml to provide HTML templates, and Compass to provide semantic CSS. Development made use of several Ruby and Rails development tools, including RSpec, Cucumber and Webrat. For dependency resolution, iDoc was recently migrated to make use of Bundler, a new Ruby system for specifying and installing libraries using the RubyGem package manager. It should be easy to install iDoc onto a system with Ruby and Xapian installed, and deployment can be automated with Capistrano. Current installations of iDoc are served up with Apache and Passenger, although again there shouldn’t be any problems with other Rails server technologies, such as Mongrel, Thin, Unicorn or even a plain old FastCGI server.

For Xapian integration, a new Ruby gem, Xapor, was developed to allow for the specification of Xapian indexes very easily through ActiveRecord. The interface is inspired very much by other popular search plugins, such as ThinkingSphinx. Xapor is currently at version 0.1.2, with the features required for iDoc currently implemented. It is intended to continue development of this gem to provide a much more sophisticated search integration with Xapian. Xapor currently also relies on the gem XapianFu, which provides an easier interface for Xapian than the plain Ruby bindings but isn’t particularly sophisticated for ActiveRecord integration. We open-sourced Xapor on Github and the gem is available on Rubygems.

Future development

There are several more features that we want to integrate into future versions of iDoc. Some of these include:

  • Notifications. It would be highly desirable to have notifications available to users. These would include things such as page additions, deletions, updates and new comments. Currently, it is planned that these would be served as an RSS feed, although other notification systems could also be added (e.g. twitter and/or email)
  • Published APIs. iDoc isn’t intended as a standalone system, and although it is easy currently to just embed help links or navigate the documentation by hand, it would be much nicer if some degree of automation could be performed.
  • An improved user login experience. Login currently relies on registration, but could be extended to include the use of other signup systems, such as Facebook Connect, Twitter OAuth and OpenID, to allow users to re-use their existing accounts on these servers rather than creating a new account.
  • Automatic generation of content sections. This is an important improvement, and would improve the navigation of documentation pages. The general principle would be to automatically provide in-page tables of contents at the top of documentation pages, and provide the ability to edit subsections rather than the entire page as a block. [Stop press: Now partially implemented and soon to be rolled out]
  • Interface customisation. While this is certainly possible currently, it requires a customiser to hand edit CSS in order to do so. It would be much nicer if there was a web-interface method to customise at least some of the interface elements for an installation, and possibly even on a per-user level at some point in the future.

As iDoc is open source, we hope that others will extend the system in useful ways.

This entry was posted in announcements, ple, projects. Bookmark the permalink. Post a comment or leave a trackback: Trackback URL.

One Trackback

  • By November 2010 progress update on November 18, 2010 at 2:30 pm

    […] provided by the use of mySparql in conjunction with a modified open source documentation system, iDoc (this is a Hedtek product).  We expect results here very […]

Post a Comment

Your email is never published nor shared. Required fields are marked *

You may use these HTML tags and attributes <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>