--- /dev/null
+<HTML>\r
+<HEAD>\r
+ <!-- Created by GNNpress/1.2 -->\r
+ <TITLE>Jigsaw design rationale</TITLE>\r
+</HEAD>\r
+<BODY BGCOLOR="white">\r
+<A HREF="http://www.w3.org/pub/WWW/" TARGET="_top_"><IMG SRC="/icons/WWW/w3c_home" WIDTH="72" HEIGHT="48"\r
+ ALT="W3C" BORDER="0"></A>\r
+<A HREF="../Overview.html" TARGET="_top_"><IMG SRC="/icons/jigsaw"\r
+ WIDTH="212" HEIGHT="49" ALT="Jigsaw" BORDER="0"></A>\r
+<H1>\r
+ Jigsaw: An object oriented server\r
+</H1>\r
+<P>\r
+This document explains the current design of <B>Jigsaw</B>. Since this project\r
+started, a year ago, the only thing that remained across the various test\r
+implementations that were written, is the choice of the Java language as\r
+the implementation language. It is not the purpose of this document to explain\r
+this choice, but we still think that having threads, garbage collection,\r
+portability and a secure language helped us a lot throughout the actual design.\r
+<P>\r
+This paper is divided into three sections, each of them describing a precise\r
+point in the space of possible servers design. We started with a simple-minded\r
+file based server, whose principal drawback was the lack of an explicit\r
+meta-information store. The second design caught up with this drawback, but\r
+lacks the ability to <I>write</I> documents to the server (in the wide sense\r
+of write, you can always implement PUT as a CGI script). The third design\r
+took on the idea of persistent objects that emerged from the second design,\r
+and brought it up to a full writable, persistent object <I>store</I>.\r
+<P>\r
+As we walk through these various design, it is left as an exercise to the\r
+reader to try to find a suitable answer to the following problems:\r
+<DL>\r
+ <DT>\r
+ Efficient content negotiation\r
+ <DD>\r
+ How can content negotiation can be efficiently implemented in this server\r
+ design. By efficient we typically mean the number of file-system accesses,\r
+ the number of files to parse, etc required before the actual content negotiation\r
+ algorithm can run.\r
+ <DT>\r
+ How does this design allow for handling PUT\r
+ <DD>\r
+ Although acknowledged by vendors only lately, the WWW was initially conceived\r
+ as a <I>read/write</I> space. Most servers can handle PUT through CGI scripts,\r
+ however, there is more to editing then just putting a file. Does the proposed\r
+ design allow you to dynamically plug in authentication information, or other\r
+ kinds of <I>meta-information</I> ?\r
+ <DT>\r
+ How does it handle configuration\r
+ <DD>\r
+ When it comes to configuration, most servers today require a stop/edit/restart\r
+ cycle. Does the proposed design allow for something better ?\r
+</DL>\r
+<P>\r
+We will see by the conclusion, that the current <B>Jigsaw</B> design is tuned\r
+for answering each of these precise questions. As we walk through the designs,\r
+we will use the configuration process of each of the designed servers as\r
+the key to enter their implementations. Configuration of servers is often\r
+left-over, although it can be used to reverse engineer the implementation\r
+of a server, as we will see.\r
+<H2>\r
+ <A NAME="file">The file-based design (or the lack of meta-information)</A>\r
+</H2>\r
+<P>\r
+The first of <B>Jigsaw</B>'s design, was aimed toward serving files, and\r
+supporting things like CGI scripts. Generally, it was the result of trying\r
+to emulate what current servers do, while still trying to take advantage\r
+of Java features (such as dynamic class loading).\r
+<H3>\r
+ Configuration process\r
+</H3>\r
+<P>\r
+In this design, the configuration process was pretty similar to classical\r
+servers. A central configuration file would split the exported information\r
+space (implemented by the underlying file-system) into various <I>areas</I>,\r
+each of them being handled in some specific ways. Typical statements of the\r
+configuration file would include things like:\r
+<PRE>/foo/bar/* FileHandler()\r
+</PRE>\r
+<P>\r
+The server would compile this statement by creating some <I>rule</I> stating\r
+that all URLs having the <CODE>/foo/bar</CODE> prefix were to be handled\r
+by some instance of the FileHandler Java class. At runtime, when receiving\r
+a request, the server engine would extract the target URL, find the best\r
+matching rule and hand out the request for processing by the appropriate\r
+<I>handler</I>. The FileHandler would do the normal work of serving the physical\r
+file mapped to the requested URL.\r
+<P>\r
+This rule mechanism was directly inspired by the CERN server, with the syntax\r
+change justified by the fact that you would be able to dynamically load new\r
+area handlers. So, for example, if the rule:\r
+<PRE>/cgi-bin/* CgiHandler()\r
+</PRE>\r
+<P>\r
+was present in the configuration file, the instance of the CgiHandler Java\r
+class would be created on demand, (i.e. only if some document in this area\r
+was requested.). This handler would be used to run the scripts in the given\r
+area, and serve their result, following the CGI specification.\r
+<P>\r
+The configuration file syntax included the concept of <I>filters</I>. Filters\r
+were invoked before the actual processing of a request by its area handler.\r
+To setup a filter on some area, you would add a statement like:\r
+<PRE>/protected/* AuthFilter("realm", CgiHandler())\r
+</PRE>\r
+<P>\r
+This functional view of filters was inspired by the Strand (see the Strand\r
+<A HREF="http://www.osf.org/www/waiba/papers/www4oreo.htm">paper</A> of the\r
+WWW'95 conference) work at OSF, which uses them in proxies rather then in\r
+servers themselves. In the above case, the request would be first run through\r
+the AuthFilter (which would check its authorization information) before being\r
+handed out to the CgiHandler. Note how the configuration file syntax allowed\r
+for passing parameters to the actual filters or handlers (such as the\r
+<I>realm</I> in the above statement). The server provided a way to specify\r
+what arguments were required by each of the handlers (or filters), and would\r
+parse them in some uniform manner, so that the configuration file syntax\r
+could be kept consistent. In spirit, this was similar to the way Apache handles\r
+configuration files, and allows each module to register for a set of\r
+configuration directives.\r
+<H3>\r
+ Lessons learned\r
+</H3>\r
+<P>\r
+This first design had some nice properties. The first one was simplicity.\r
+Not so much simplicity <I>per-se</I>, but simplicity because the design would\r
+match what people expected from a web server. We felt that the ability to\r
+run the server straight out of the box, without having to deal with each\r
+exported information individually was important (i.e. the ability to set\r
+global defaults, that would apply to everything in the exported information\r
+space).\r
+<P>\r
+From an implementation point of view, having only one handler object per\r
+area also meant that the server would not be overcrowded with thousands of\r
+small objects representing each exported resource. This meant that memory\r
+requirements would be kept low (well, at least proportional to the number\r
+of areas, which is expected to be much smaller then the number of exported\r
+objects).\r
+<P>\r
+However, the big win in this design was the integration of the concept of\r
+filters inside the server architecture. This proved invaluable as a mean\r
+of decoupling server's functionality. The simple fact that authentication\r
+could be handled in a more generic framework, for example, brought a lot\r
+of simplicity in the server design. The important thing here was that the\r
+number of phases to process a request was significantly reduced. Instead\r
+of being:\r
+<OL>\r
+ <LI>\r
+ URL translation to a local name\r
+ <LI>\r
+ Access authentication\r
+ <LI>\r
+ Compute the reply\r
+ <LI>\r
+ ...\r
+</OL>\r
+<P>\r
+We would now have:\r
+<OL>\r
+ <LI>\r
+ URL translation\r
+ <LI>\r
+ Compute the reply\r
+ <LI>\r
+ ...\r
+</OL>\r
+<P>\r
+Since the authentication phase would be integrated (through filters) to the\r
+computation of the reply stage. Reducing the number of phases in request\r
+processing also meant that the general request processing model of the server\r
+would be more simple, which in turn meant that extension writers would have\r
+their job eased. Two other important benefits of filters were:\r
+<OL>\r
+ <LI>\r
+ The ability to extend the server functionality <I>orthogonally</I> to the\r
+ way the request was actually processed by its handler.\r
+ <LI>\r
+ The cost of running the filters would be paid <I>only</I> when appropriate.\r
+ It allowed you for example, to specify that the <CODE>/foo/bar</CODE> should\r
+ be enhanced with some special logging.\r
+</OL>\r
+<P>\r
+However, there were bad news too. Things began to turn out really badly when\r
+we tried to add content negotiation to this design. There was mainly two\r
+ways to proceed:\r
+<DL>\r
+ <DT>\r
+ The CERN server way\r
+ <DD>\r
+ Once the requested URL has been converted to a file-system path, check if\r
+ it comes with an extension. If no extension is available, list all the files\r
+ in the requested document's directory, and match them against the root of\r
+ the file name. Then run the negotiation process among the set of matching\r
+ entries.\r
+ <DT>\r
+ The index file way (currently implemented by Apache and WN)\r
+ <DD>\r
+ For a document to be negotiated, we have to write some <I>map</I> or\r
+ <I>index</I> file in that directory, describing each variant of the negotiated\r
+ resource. When needed, the server parses this file, and negotiates among\r
+ the set of described variants\r
+</DL>\r
+<P>\r
+The CERN server approach was quite nice since it would run behind the scene:\r
+the web master would just start its server, and without (nearly) any additional\r
+information, the server would support content negotiation. However, the\r
+efficiency of the above scheme was obviously poor. The second approach coped\r
+with the efficiency problem (although the server would still have to parse\r
+an ASCII file to get the set of variants), but incurred more work for the\r
+web master (which would now have to write the appropriate description files).\r
+<P>\r
+Although we felt content negotiation was an important feature to provide,\r
+what this revealed was a much more general problem: <B>the lack of an efficient\r
+meta-information store</B>. With such a thing, we would be able to efficiently\r
+access a resource's meta-information, which would allow for a fast implementation\r
+of the content-negotiation algorithm.\r
+<P>\r
+To make this problem more apparent, ask yourself how your favorite server\r
+deduces the content-type of the document it serves. For example, the CERN\r
+server walks through all file extensions it knows of (which it gets from\r
+the central configuration file), trying to find the one that matches the\r
+name of the document (a file in this case) to be served. The thing\r
+that matters here is not really the computation of the content-type by itself,\r
+but rather the fact that it is <I>recomputed</I> each time the same document\r
+is served. If your document is accessed fifty times per second, then you\r
+will walk through the extensions list fifty times, in the same second.\r
+<P>\r
+For content-type this is not a big deal, although if the list of known extensions\r
+grows too much, then this can cause some significant damage to the server\r
+performances. Where things really hurt, is when it comes to meta-information\r
+that is hard to compute, such as digital signatures, or meta-information\r
+to be extracted by parsing an HTML document (i.e. as given through the\r
+<CODE>META</CODE> tag). In these cases, you really don't want the server\r
+to spend time recomputing the meta-information, what you want is to cache\r
+them once they have been computed for latter reuse.\r
+<P>\r
+This lack of efficient meta-information store has already been acknowledged\r
+by some server authors. The WN server [reference], for example, defines\r
+per-directory configuration files, that allows web-masters to provide the\r
+additional meta-information pertaining to specific documents in some given\r
+directory. We will come back to this latter.\r
+<H3>\r
+ Summary\r
+</H3>\r
+<P>\r
+At this point, we had a first server running, but we were stuck, mainly by\r
+the lack of a meta-information store. We had also left over a number of important\r
+features, such as how PUT would be handled, how a single file would be protected\r
+(although this would be possible as an extreme case of an area containing\r
+a single file).\r
+<P>\r
+In fact, we were ready for our next redesign.\r
+<H2>\r
+ <A NAME="directory-config">Efficient per-directory configuration files (or\r
+ the read-only server)</A>\r
+</H2>\r
+<P>\r
+Guess what ? The second redesign of <B>Jigsaw</B> emphasised the meta-information\r
+storage. Our first goal was to set for a real database of meta-information\r
+about resources. Numerous ways were available for implementing this, ranging\r
+from connecting the server to a full-blown database, down to simple per-directory\r
+configuration files. The WN server uses the latter approach with some success,\r
+so we first settled for this.\r
+<H3>\r
+ Toward an efficient meta-information store\r
+</H3>\r
+<P>\r
+The first question we had to answer was <I>how</I> the meta-information store\r
+would be implemented. The WN implementation offered at least one possible\r
+solution. Each meta-information would be written down in a per-directory\r
+configuration file by the web master (with the optional help of an\r
+<I>indexer</I> program). After the translation process (i.e once the URL\r
+was converted to a local name), the server would check the appropriate\r
+per-directory configuration file, parse it and get from it the required\r
+information (e.g. the content-type of the document to be served, it's title,\r
+etc.).\r
+<P>\r
+We were not quite happy with this model, for several reasons. The first obvious\r
+one was the CPU cost of reading this per-directory configuration file on\r
+each request. Although this can be less expensive than recomputing entirely\r
+the meta-information from scratch, we felt that the server had better things\r
+to do than parsing some ASCII file before being able to handle an incoming\r
+request. We felt this particularly important since we were to use a\r
+multi-threaded server rather than a forking server, which meant that we could\r
+- at least - be able to cache the result of parsing the per-directory\r
+configuration file (more on this latter).\r
+<P>\r
+The other concern we had, which was consistent with our first experiment,\r
+was to try to reduce the number of phases needed to process a request (with\r
+the hope that this would make the overall design simpler, which ultimately\r
+would allow us to provide a simple extension API). In particular, we were\r
+interested in having an <I>object oriented lookup</I> rather than the not-so-nice\r
+translation phase of common servers. What we really wanted to achieve here,\r
+is a simplification of the server main loop, so that it would be captured\r
+by something similar to the following pseudo code:\r
+<CENTER>\r
+</CENTER>\r
+<CENTER>\r
+ <TABLE BORDER CELLPADDING="2">\r
+ <TR>\r
+ <TD><PRE>public class httpd extends Thread {\r
+\r
+ public void run() {\r
+ while ( true ) {\r
+ Socket client = accept() ;\r
+ new httpdClient(socket).start() ;\r
+ }\r
+ }\r
+}\r
+\r
+public class httpdClient extends Thread {\r
+\r
+ public void run() {\r
+ while (keep_alive) {\r
+ Request request = getNextRequest() ;\r
+ Resource target = lookup(request) ;\r
+ Reply reply = target.perform(request);\r
+ reply.emit() ;\r
+ }\r
+ }\r
+}\r
+</PRE>\r
+ </TD>\r
+ </TR>\r
+ </TABLE>\r
+</CENTER>\r
+<PRE>\r
+</PRE>\r
+<P>\r
+Once again, our goal here was to provide a simple enough request processing\r
+model, to allow for an easy to use server-side API. \r
+<P>\r
+Finally, in the process of architecturing this new design, another idea made\r
+came about: as the server would have to maintain a database of per-resource\r
+information, why not use this database to also <I>configure</I> the resource\r
+behavior. A resource would have a set of <I>attributes</I>, containing both\r
+its meta-information, and its configuration data. As an example, the FileResource\r
+(i.e. the resource class that knows how to serve files) could have a\r
+<CODE>writable</CODE> attribute to configure the fact that PUT is allowed.\r
+<P>\r
+As a result of all this, the idea that <B>exported resources should be persistent\r
+objects</B> seemed appealing enough to be worth a try.\r
+<H3>\r
+ Configuration process \r
+</H3>\r
+<P>\r
+Instead of going into the implementation details, we will quickly walk through\r
+the configuration process, giving the details needed to understand the pitfall\r
+of this design. To implement persistent objects, we first needed a way of\r
+describing these objects, so that the web master would be able to create\r
+and configure them. We choosed to use per-directory configuration file, along\r
+with an ASCII syntax for describing these objects. These files would then\r
+be read and parsed by an <I>indexer</I> program, that would resurrect them\r
+out of their ASCII representation, and dump them out in a binary format,\r
+suitable to be fed to the server itself. The main purpose of this extra\r
+compilation stage was to release the server from having to parse the ASCII\r
+files as mentioned above.\r
+<P>\r
+In this setting, a typical configuration file would look like this:\r
+<CENTER>\r
+</CENTER>\r
+<CENTER>\r
+ <TABLE BORDER CELLPADDING="2">\r
+ <TR>\r
+ <TD><PRE> 1] (entity w3c.http.core.DirectoryEntity<BR> 2] :name "root"\r
+ 3] :entities ((entity w3c.http.core.FileResource \r
+ 4] :name "Welcome.html"\r
+ 5] :content-type "text/html")<BR> 6] (filter w3c.http.auth.BasicAuthFilter<BR> 7] :realm "my-realm"<BR> 8] (entity w3c.http.core.FileResource \r
+ 9] :name "protected.html" \r
+10] :content-type "text/html"))))\r
+</PRE>\r
+ </TD>\r
+ </TR>\r
+ </TABLE>\r
+</CENTER>\r
+<P>\r
+<BR>\r
+This file would be put in some directory, then the web master would run the\r
+<CODE>jindex</CODE> program, that would convert this file (usually named\r
+<CODE>jindex.txt</CODE>) into a binary file, containing a serialization of\r
+each of the objects described in it (this last file would be called\r
+<CODE>jindex</CODE>). More precisely, the above file describes three different\r
+resources:\r
+<UL>\r
+ <LI>\r
+ The first one (starting at line 1), of Java class DirectoryResource describes\r
+ how to export the directory in which the above configuration file was found.\r
+ It has two attributes: its <I>name</I>, whose value here is <B>root</B>,\r
+ and an <I>entities</I> attribute giving the list of sub-resources (lines\r
+ 3 to 10)\r
+ <LI>\r
+ The first sub-resource (lines 3 to 5) described here is a simple file resource,\r
+ whose name is <B>Welcome.html</B> and whose content-type is <B>text/html</B>.\r
+ In the URL space, this resource would be named\r
+ <CODE>.../root/Welcome.html</CODE>. The lookup process would first get to\r
+ the resource named <CODE>root</CODE>, and would invoke the root resource\r
+ <CODE>lookup</CODE> method, with as parameter the Welcome.html string, to\r
+ get back the actual FileResource encapsulating the file.\r
+ <LI>\r
+ The last sub-resource (lines 6 to 10) here is filtered by an instance of\r
+ the BasicAuthFilter class. This class has a <I>realm</I> attribute indicating\r
+ in which realm incoming requests should be authentified, before being handed\r
+ out to the actual resource.\r
+</UL>\r
+<P>\r
+Once this file was compiled, the server would then be run. Upon receiving\r
+a request the lookup process would load into memory the appropriate binary\r
+files, and resurrect the objects as required. Loading these files was done\r
+through a cache, so that efficient access would be allowed (we'll come back\r
+to this latter).\r
+<P>\r
+With this new design, we were eager to see how content negotiation would\r
+be handled (hey, the whole point was to have an efficient implementation\r
+of content negotiation). Content negotiation would be handled by the\r
+NegotiatedResource Java class, that would store all its variants, and their\r
+meta-information, in order to run the negotiation algorithm as quickly as\r
+possible. A typical negotiated resource would be described by:\r
+<P>\r
+<CENTER>\r
+ <TABLE BORDER CELLPADDING="2">\r
+ <TR>\r
+ <TD><PRE> 1] (entity w3c.http.core.NegotiatedResource<BR> 2] :name "foo"\r
+ 3] :variants ((entity w3c.http.core.FileResource \r
+ 4] :name "foo.gif" \r
+ 5] :type "image/gif" \r
+ 6] :quality 0.9)<BR> 7] (entity w3c.http.core.FileResource \r
+ 8] :name "foo.png" \r
+ 9] :type "image/x-png" \r
+10] :quality 1.0)))\r
+</PRE>\r
+ </TD>\r
+ </TR>\r
+ </TABLE>\r
+</CENTER>\r
+<P>\r
+In brief, the NegotiatedResource class had a <I>variants</I> attribute describing\r
+all the variants among which negotiation was allowed for the given resource.\r
+Each variant would get its quality from a <I>quality</I> attribute. Note\r
+how this time, all the information required to run the content negotiation\r
+algorithm would be readily available to the server. Note also how the\r
+content-type and other meta-information would be accessed directly once a\r
+resource was restored from its serialized form.\r
+<P>\r
+However, a set of new problems arose in this design, which made up a new\r
+lesson.\r
+<H3>\r
+ Lessons\r
+</H3>\r
+<P>\r
+First of all, we were quite happy with this new design. It had solved the\r
+meta-information store problem in a nice way, but most interestingly, each\r
+exported resource would now be encapsulated in its own object, bringing up\r
+a natural server extension API: to extend the server with a new functionality\r
+(such as CGI scripts handling), you would just write a new resource sub-class\r
+matching your needs. One nice side-effect of this new design was that agents\r
+(eg downloading code to the server) could be implemented really easily: instead\r
+of serializing a resource to the disk, you would just serialize it to a socket\r
+connected to the target server . The implementation of this design indeed\r
+came with an Agent class, sub-class of the generic Resource class, that would\r
+allow them to <I>go</I> from one site to the other by the use of a <CODE>go(URL\r
+url)</CODE> method.\r
+<P>\r
+The filter concept was reintegrated in the new design. This was not so simple:\r
+the previous design allowed you to filter a whole area (as defined by its\r
+prefix), what we wanted now was both the ability to plug a filter on\r
+<I>one</I> single resource, while at the same time keep a way to tell that\r
+a whole area should be filtered. To solve the conflict, we wrote a new\r
+FilteredResource class, whose lookup method would invoke any of its filters.\r
+By using this trick, we were able to filter, for example, all accesses to\r
+some <CODE>/foo/bar/*</CODE> area: the resource exporting the\r
+<CODE>bar</CODE> directory would be a sub-class of the FilteredResource,\r
+and what ever filters were plugged on it would be called each time the bar\r
+resource was crossed by the lookup process.\r
+<P>\r
+Compared to other servers, this design allowed for an efficient implementation,\r
+since the cost of recomputing meta-information would totally disappear. There\r
+were still, however, some drawbacks.\r
+<P>\r
+The implementation complexity reached a new order of magnitude: this time,\r
+the server process would have to host one Java object per exported resource.\r
+This meant that it might have to keep in memory hundreds of thousands of\r
+these objects. Fortunately, because these objects were persistent, it was\r
+quite easy to fix this problem by using some appropriate caching mechanism.\r
+Once a resource was detected idle for a given duration of time, it could\r
+be serialized back to disk (if needed), freeing the memory it occupied before.\r
+It would then be brought up to memory later if requested.\r
+<P>\r
+The configuration process was now much more difficult to handle. We experimented\r
+with ways of making it easier. The idea here, was to put the complexity in\r
+the indexer program: when the indexer was run in some directory, it would\r
+match all files found against a global configuration database, describing\r
+how files should be wrapped into resources, based on their extensions. A\r
+typical statement in this configuration database would look like:\r
+<CENTER>\r
+</CENTER>\r
+<CENTER>\r
+ <TABLE BORDER CELLPADDING="2">\r
+ <TR>\r
+ <TD><PRE>(extension ".html"\r
+ :class w3c.http.core.FileResource\r
+ :type "text/html"\r
+ :quality 1.0\r
+ :icon "html.gif")\r
+</PRE>\r
+ </TD>\r
+ </TR>\r
+ </TABLE>\r
+</CENTER>\r
+<P>\r
+When encountering a .html file, the indexer would then know that by default\r
+(i.e. if no other statement was made about the file in the per-directory\r
+configuration file), it should build a FileResource to export the file.\r
+<P>\r
+Another problem with this design was the requirement to run the indexer in\r
+each directory before running the server. This was definitely breaking with\r
+our "run straight out of the box" goal. More importantly, the fact that\r
+meta-information were stored <I>both</I> in a ASCII file version, and in\r
+a binary version (the compiled form) meant that the server had to deal with\r
+two kinds of inconsistencies:\r
+<UL>\r
+ <LI>\r
+ As the server would cache the binary files, they might become out of\r
+ synchronization with the ASCII file, or even with the binary file themselves:\r
+ we lacked a way of notifying the server that it should flush some cached\r
+ binary file (well, it was possible, but added as a hack, instead of being\r
+ thought of right from the beginning).\r
+ <LI>\r
+ We wanted our server to handle PUT of new documents. This would mean that\r
+ the server itself would have to edit the <I>binary</I> version of the\r
+ per-directory configuration file. This, by itself was easy, but changes to\r
+ the binary file would be superseded by the recompilation of the ASCII\r
+ per-directory configuration file.\r
+</UL>\r
+<H3>\r
+ Summary\r
+</H3>\r
+<P>\r
+Except for the problem of the cache coherency mentioned above, this design\r
+was quite nice for a read-only server. In particular, it made us buy the\r
+idea of a server of persistent objects, by opposition of the simple file-based\r
+design that we designed first. This was a big step forward.\r
+<P>\r
+However, when it came to make the server writable, things hurt. The real\r
+lesson from this design was two fold:\r
+<UL>\r
+ <LI>\r
+ Persistent objects were good\r
+ <LI>\r
+ If you were to have your server serve an <I>editable</I> space, then simply\r
+ dumping the body of a PUT method into some file was not enough: you ought\r
+ to provide a way of <B>editing the meta-information associated to each\r
+ resource</B> through the server itself.\r
+</UL>\r
+<P>\r
+As a side effect, the second point would solve our caching of the\r
+per-directory information problem: <I>only</I> the server should write these\r
+information, in order to avoid the conflicts created by having two potential\r
+sources of edits to a single resource. The implications of these two lessons\r
+lead us to a new redesign of <B>Jigsaw</B>.\r
+<H2>\r
+ <A NAME="object-oriented">The object oriented design</A>\r
+</H2>\r
+<P>\r
+This last presented design is the current <B>Jigsaw</B> design, as of version\r
+1.0a. The main lessons it takes into account can be summarized as:\r
+<UL>\r
+ <LI>\r
+ All exported resources are persistent objects, they can be edited only through\r
+ the server\r
+ <LI>\r
+ All resources should be able to support filters\r
+ <LI>\r
+ The configuration process must be changed in order to be done through the\r
+ server, so that it can keep its cache of persistent resources up-to-date.\r
+</UL>\r
+<P>\r
+As usual now, we are going to walk through the configuration process, trying\r
+to highlight the parts of implementation that are relevant to this\r
+discussion.\r
+<H3>\r
+ Configuration process\r
+</H3>\r
+<P>\r
+As of this release, the configuration process is now totally runnable through\r
+HTML forms. To handle this in some generic fashion, the resource objects\r
+have been extended with a way to describe themselves. Each resource supports\r
+a <CODE>getAttribute()</CODE> method, which is responsible for returning\r
+the full list of attributes they support, along with various type information.\r
+More precisely, each declared attribute of a resource as a piece of\r
+meta-information associated with it, that describes how values for this attribute\r
+can be edited and saved. Based on this description, the <B>Jigsaw</B>\r
+configuration module is able to dynamically generate a form, suitable to\r
+edit any resource attribute. This is how <B>Jigsaw</B>'s configuration is\r
+done today.\r
+<P>\r
+Resource filters are handled as a specific attribute of the FilteredResource\r
+class, whose instances can be edited dynamically. As a side effect of this\r
+resource module, it appeared that <I>any</I> piece of data (be it configuration\r
+or not), that needs to be edited through the server could now be turned into\r
+some specific kind of resources. The current <B>Jigsaw</B> implementation\r
+takes full advantage of this, and things like authentication realms are typically\r
+implemented as special classes of resources. This not only means that we\r
+get authentication realms for free, it also means that any caching scheme\r
+applied to resources also applies to authentication realms. <B>Jigsaw</B>\r
+keeps an LRU list of resources loaded in memory, this list can include things\r
+like authentication realms (which are kept in memory only when they are really\r
+needed), authentication realm users, etc. All these data structures, by\r
+inheriting from the Resource class, gets the caching behavior for free.\r
+<P>\r
+Another place where resources are used is in the global configuration database.\r
+Remember how the previous design maintained a list of extensions, in order\r
+to provide to the indexer program the knowledge about how files should be\r
+indexed (wrapped into resource instances) by default. This last design of\r
+<B>Jigsaw</B> maintains a similar database, with the indexer now being part\r
+of the server itself. For example, each extension property is defined through\r
+a special <I>extension resource</I>, that has attributes for storing the\r
+class of resource to export the matching files, default attribute values\r
+(such as the expire date, etc).\r
+<P>\r
+Because the indexer is now part of <B>Jigsaw</B>, the server can now be used\r
+straight out of the box: you can just run <B>Jigsaw</B> in some root directory,\r
+and it will dynamically create the queried resource instances as it runs.\r
+Each time a resource is created, it is made persistent, so that next time\r
+the server runs, it doesn't have to recreate them. Resources are serialized\r
+into per-directory binary files, which are totally under the control of the\r
+server (these files are usually named <I>.jigidx</I> files, with their backup\r
+version being <I>.jigidx.bak</I>).\r
+<P>\r
+The usual stop/edit/restart cycle needed to change the server configuration\r
+has now totally disappeared, you can edit the resource configuration (defined\r
+through some of their attribute values) dynamically, while the server is\r
+running.\r
+<H3>\r
+ Lessons\r
+</H3>\r
+<P>\r
+<B>Jigsaw</B>'s current design seems to fulfill all our needs. The implementation\r
+still needs some tuning, and some features need to be added, but the APIs\r
+seems pretty stable now.\r
+<P>\r
+The implementation is now fairly complex. It uses a number of caching mechanisms\r
+(who said a server was just a series of caches ?), which do impact the current\r
+extension APIs (although not the one that will be commonly used, such as\r
+the resource class). Improvement to the implementation will be made by turning\r
+most of the design into <I>interfaces</I> and by making the current\r
+implementations (the actual classes) a set of sample implementations for\r
+these interfaces.\r
+<P>\r
+<B>Jigsaw</B> also lacks some important features: support for virtual hosts,\r
+server-side markup and these kind of things. Most of these features (if not\r
+all) can be implemented as either new resource classes, or new filters.\r
+<H2>\r
+ Next step\r
+</H2>\r
+<P>\r
+So what will the next design of <B>Jigsaw</B> will be? Well, as said above,\r
+there will probably <I>not</I> be a redesign of <B>Jigsaw</B> until some\r
+time. Once the current implementation is cleanly separated between a set\r
+of interfaces and a set of implementations, it will be possible to add\r
+functionality in a clean way.\r
+<P>\r
+Among the number of things that might happen, there are two of them on which\r
+I would like to insist:\r
+<UL>\r
+ <LI>\r
+ As a matter of fact, <B>Jigsaw</B> can be considered as some sort of object\r
+ oriented database. It would be nice to be able to reuse available technology\r
+ here. The ResourceStore interface can be implemented to fetch serialized\r
+ objects from any kind of database, rather then from a per-directory configuration\r
+ file. This could improve both the robustness and the performance.\r
+ <LI>\r
+ The configuration process, as described above, suffers from a lack of user\r
+ friendliness. This can be remedied by having the <I>client</I> being more\r
+ smart. The use of HTML forms as a general means to edit the object attributes\r
+ limits the range of usable UI metaphors. Work will be done to first provide\r
+ access to the server through RMI, and then using this remote interface, write\r
+ up a real server administration application (probably composed of a set of\r
+ applets).\r
+ <LI>\r
+ Use Java ability to move code around. All along the paper we have insisted\r
+ on the fact that resources were self-contained: they know of their own\r
+ description, their configuration and their behavior. This means that if you\r
+ were able to move these objects on the network, you could achieve a nice\r
+ replication environment, bringing computation closer to the clients. More\r
+ precisely, today the world is divided into two big categories: the\r
+ CGI world which emphasis on the computation happening on the server side,\r
+ and the Applet world, which emphasizes on the computation happening on the\r
+ client side. We would like to experiment with something in between those,\r
+ where computation would happen in proxy servers.\r
+</UL>\r
+<H2>\r
+ Conclusion\r
+</H2>\r
+<P>\r
+We have walked through three different server designs. The first one was\r
+a simple file-based server, we show why the lack of a meta-information store\r
+lead to an inefficient server implementation. The second design corrected\r
+this by using per-directory configuration files. This effectively solved\r
+the first problem, however it brought up a new set of issues when it came\r
+to have a <I>writable</I> server. Our last design is an attempt at solving\r
+all the problems encountered so far, it has now been stable for several months.\r
+<P>\r
+It is interesting to note how our requirements on the server have changed\r
+during the whole year, in particular, the emphasis on a server to both serve\r
+and <I>store</I> documents has taken an increasing priority in the design.\r
+<P>\r
+To conclude, I would like to highlight how the three problems we mentioned\r
+in the introduction are now solved:\r
+<DL>\r
+ <DT>\r
+ Efficient content negotiation\r
+ <DD>\r
+ We show that this first problem is related with the fact that our initial\r
+ design didn't support efficiently a meta-information store. The current version\r
+ of <B>Jigsaw</B> supports content negotiation and is able to run it\r
+ <I>without</I> touching the disk in most cases.\r
+ <DT>\r
+ PUTed document meta-information\r
+ <DD>\r
+ <B>Jigsaw</B> supports PUT: the first time a document is PUT'ed to the server\r
+ (it is created in fact), <B>Jigsaw</B> creates an appropriate empty resource\r
+ instance, which will capture all the meta-information that come from the\r
+ PUT request. This means that you can safely PUT a document whose name is\r
+ foo.bar, and whose content-type is text/html.\r
+ <DT>\r
+ Editing the information space configuration\r
+ <DD>\r
+ Adding authentication to an area of information exported by the server, is\r
+ just adding an authentication filter to the appropriate DirectoryResource.\r
+ This can be done entirely through forms, by editing the resource's attributes.\r
+</DL>\r
+<P>\r
+ <HR>\r
+<BR>\r
+<I><A HREF="mailto:jigsaw@w3.org">Jigsaw Team</A><BR>\r
+$Id: wp.html,v 1.1 2010/06/15 12:20:10 smhuang Exp $</I>\r
+</BODY></HTML>\r
projects / Benchmarks_CSolver.git / blobdiff