Pilaster PHP Library in GitHub

Over the holiday break, I decided to dust off the Pilaster document database and bring it up to some acceptable level of stability.

As it stands now, Pilaster is a document database that provides similar capabilities to that of MongoDB. It is nowhere near as performant as MongoDB -- then again, it's written in pure PHP and does not require a separate application.

I figured it would be useful for smaller system or for applications running on hosted platforms.

In the backend, it uses Lucene (or, rather Zend's port of Lucene) to provide a high(-ish) performance datastore. You can search it using either name/value arrays or search strings. Yes, search strings are the database's query language. Weird... I know.

To test it out (and contribute, if you'd like), take a look at the Git repository here:

http://github.com/technosophos/Pilaster

If you want to start with a build of it, you can download 2.0-alpha1 (or whatever today's latest version is) by going here:

http://github.com/technosophos/Pilaster/downloads

Versions

When I initially developed Pilaster, it was basically a form of the Sinciput and Rhizome projects. It worked, but it was very tied to the way those systems used data. That was Pilaster 1.x.

As I took a fresh look at Pilaster in December, I realized that I could make it much more generic -- in short, I could make it work with native data structures! With minimal refactoring, I changed the entire Pilaster data model.

I decided that would be Pilaster 2.x.

Pilaster is not quite stable. The unit tests for the main public API all pass. However, the low-level driver is not complete. Import and export still need to be added. I am also uncertain as to how stable the Zend index really is. Your testing can help me discover what works and what doesn't.

Full Search Added to Rhizome

Yesterday I checked in what I think will be the last major change in Rhizome before I release the 1.0 version.

What was the feature? Full text search. Actually, full text search has always been supported, but it required Rhizome implementors to do lots of the heavy lifting on their own. That has now changed.

Now, from a Rhizome IndexSearcher instance, you can use the simpleSearch() methods to fire off very complex searches without having to do any custom coding.

SimpleSearch supports the following:

• Specifying which metadata should be searched
• Determining whether or not the main body should be searched
• Returning results in sorted order by score
• Handling multiple pages of search data
• Using the ProxyRhizomeDocument class to return certain fields immediately, while delaying the load of entire documents until it is necessary
• Handling very complex search strings, including metadata-specific searches.

On that last item, an example is in order. You could, for example, search for all documents that have a specific tag (or label) and contain a specific word. Here's the search string:

+tag:Nietzsche +zarathustra

This would search for only items with the Nietzsche tag, and with the word zarathustra somewhere in the body or metadata. A whole batch of operators and so on are supported, thanks to the Lucene subsystem.

Rhizome and the PAC Pattern

Rhizome is designed basically according to the PAC (Presentation-Abstraction-Control) design pattern. PAC is similar to MVC, but probably better for web development, and definitely better for Rhizome's goals.

Presentation:
The presentation layer bundled with Rhizome, which makes use of Velocity, is not terribly impressive. It does what a presentation layer ought to do, and no more. It is responsible for taking data and formatting it for return to the user.

Rhizome is pretty flexible on the presentation layer. Velocity is included because I like it enough to use it, and don't hate it enough to look for something else. Also, its introspection abilities are excellent, and make template writing much easier. No special interfaces, etc.

Of course, all that it takes to use a different presentation layer is a few lines of code extending from com.technosophos.rhizome.command.AbstractCommand. And a couple of lines of XML in commands.xml.

But one thing should be noted about the presentation layer as it is: No (non-GUI related) processing is done in the presentation layer. No callbacks to the system are used, and no new resources are retrieved from the system once template rendering is initiated. In short, presentation is passive. It takes what it's given, and it presents it. That's all.

Abstraction:
This is similar to the Model layer in MVC, but in PAC it is the layer responsible for managing the data.

Rhizome has a robust abstraction layer, packaged in com.technosophos.rhizome.repository. This abstraction provides unified methods of accessing data. The current implementation uses a disk-based repository and indexes.

The abstraction engine is broken into three parts: A repository (or database) for storing entire documents, a searcher for finding resources in the repository (usually by search data and metadata), and an indexer. The indexer takes documents and preprocesses them before storing them in the document repository.

Basically, the abstraction layer of Rhizome is a document database (DocumentDB). It is responsible for storage and retrieval of documents.

In some ways, the Rhizome document DB is itself a PAC: The RepositoryManager manages the repository like a controller. The search, indexing, and repository access tools deal directly with data abstraction. And the document representations (in com.technosophos.rhizome.document) providing a common format for the app's client -- in this case, Rhizome's controller. Thus, the RepositoryManager retrieves content, makes sure it is in the correct format, and then returns to the RhizomeController. (Is this a valid instance of PAC? Probably. PAC controllers can be chainable.)

Control:
This layer is responsible for exerting control over the rest of Rhizome. All requests pass through the controller, and the controller is responsible for passing requests to both the repository (abstraction) and whatever the presentation layer is (Velocity, for example).

The com.technosophos.rhizome.controller tree -- and the RhizomeController object specifically -- handle control of the program. RhizomeController is a Front Controller (Which seems to be a compatible pattern with PAC) that can be chained to another controller. Usually, the external controller is a Servlet.

The controller maps requests (from the external controller or user) to a chain of internal commands that must be run in sequence to fulfill the request.

Rhizome Status

Rhizome is the new backend that will drive Sinciput (it is a complete replacement of Pilaster, which was a good, but limited, architecture).

In a nutshell, Rhizome is a non-hierarchical storage and retrieval engine that includes full text searching, metadata-based data retrieval, relationship support, and an open extension mechanism.

While Pilaster was limited to a single sort of data storage mechanism (namely, Berkeley DB), Rhizome has no such limitation. Further, while Pilaster required that the repository and the index files be located in the same "place", Rhizome makes no such requirement.

As it stands now, Rhizome has the following items completed already:

• A handful of classes implementing the new Rhizome XML format.
• A core set of managing classes and interfaces.
  
• A file system implementation of the repository.
• A Lucene-based implementation of the search indexing code.

The main task left, now, is to implement the searching interface for Lucene, which will provide a method of performing either semi-structured or completely structured searches of the Rhizome repository.