SVG Exercise 2: Plotting a Timeline, Part 1

In this two-part homework exercise, we will write XQuery to output SVG in the form of a timeline, working with our Digital Mitford collection of letters, coded in TEI. The end result of this first exercise will look something like this (with the colors, text positioning, fonts, and styling up to you). After the first exercise, we'll have plotted the line on the left with the circles inside. You will also learn how to create and save a directory of your XQuery scripts on our eXist database, and to save your output results to the eXist database to view in a browser. In the second exercise, we will plot more information about specific letter correspondents on the timeline framework we begin here.

Here’s how to locate the Mitford letters collection in our newtFire eXist-db: collection('/db/mitford/letters').

Namespace declarations: Output SVG from input TEI

In order for us to build functional XQuery that reads TEI and outputs SVG, we need to declare two namespaces:

• The default element namespace will control the output, so we will define the SVG namespace here.
• Since we are reading from the TEI, we also need to distinguish its namespace, so we will add another namespace line that we will reference with the tei: namespace prefix in our XQuery code.

Here are the two namespace lines that we need:

      declare default element namespace "http://www.w3.org/2000/svg"; 
      declare namespace tei="http://www.tei-c.org/ns/1.0";      
      

Our task

Our task with this homework assignment is to plot in SVG a timeline representing with a hashmark each year, and providing space for 365 days in between. Ultimately we want to be able to position specific letter dates on this timeline or mark specific date ranges, like 1819-01-12 to 1822-06-23, in reference to our timeline. For right now, though, we want to concentrate on plotting each year in a set interval along a vertical line. Once we plot the dates, we want to retrieve a count of letters written in a given year, and output a circle of a size determined by the count of the letters. We will also output text in SVG to mark each year and record the counts of coded letters in the collection for each year.

Working with global variables

We are going to bootstrap some of this for you to show you some new things in XQuery, so you can see how global variables are defined and so we can concentrate on outputting SVG in this homework exercise. Global variables are available throughout an XQuery document, inside its nested queries, at any level, so when you define them, you can call them anywhere where you need them. Think of them as similar to Let statements in a FLWOR, but Let statements define a condition that lasts only inside a particular FLWOR and if you need to call them in a new FLWOR at another point in your XQuery, you would need to define them again.

We are giving you some XQuery code written for the Digital Mitford project to begin. Notice how we have set some global variables to read the Mitford collection and reach into its TEI header. What we are doing here is grabbing the official dates that we set deep in the TEI headers of the Mitford files. These are formatted as yyyy-mm-dd, and we are tokenizing the date string on the hyphen (-) to return just the first token, which is the four-digit year. (You will find yourself needing to apply this tokenize function to compare and evaluate years throughout the assignment.)

         xquery version "3.0";
declare default element namespace "http://www.w3.org/2000/svg"; 
declare namespace tei="http://www.tei-c.org/ns/1.0";

declare variable $mitfordColl := collection ('/db/mitford');
declare variable $lettersColl := collection('/db/mitford/letters');
declare variable $letterFiles := $lettersColl/*;
declare variable $letterDates := $lettersColl//tei:teiHeader//tei:msDesc//tei:head/tei:date/@when/tokenize(string(), '-')[1];

let $distinctYears := distinct-values($letterDates)
for $distinctYear in $distinctYears
order by $distinctYear
return $distinctYear
              
      

Notice how we have set up a list of global variables, followed by a simple FLWOR with a for loop. Go ahead and copy this into your eXist window, run it and view the results. We begin here because we want to survey how long a time span we will be plotting in our SVG, so this will give us just a handful of distinct years in order. If you like, add a few more diagnostic variables to return, say, a count of the letter files or anything else that interests you. Remember that you need to use the tei: namespace prefix before every TEI element, but not the attributes.

Plotting the timeline

In plotting a vertical line that runs from top to bottom in chronological order, we take advantage of the y-coordinate space that increases as we move down the screen with SVG. First of all, we need to know how long our line should be. To measure it, remember that we want to mark a small set of years separated from each other by 365 days. We need to write variables to determine how many years we need to plot, and then separate them by an interval of 365. We could do this by hand, and pound this out point by point, but since our letters collection is going to change as more and more letters are coded, it would be better to write code that searches for the maximum and minimum year represented in the collection at any given time. You should to use the XPath min() and max() functions to define variables identifying the earliest and latest years in our series.

How will you determine the length of the line? Define variables to determine the number of years to plot (a simple subtraction). To stretch out the years by 365 days or pixels apart, you need to multiply by 365. (Note: An XQuery variable can hold the results of a simple arithmetic calculation. The operators +, -, *, and div are used for addition, subtraction, multiplication, and division. We can't use a forward slash for division because that has an XPath meaning.) At some point in this process, you will need to convert the year strings into integers in order to do basic calculations. In XQuery, you must do this by wrapping the xs:integer() function around your code holding a year.

Now that you have defined the variables you need to measure the line, we can begin plotting SVG! This is much like what we have done with plotting an HTML file around an XQuery script. The only difference is that you are writing SVG, and you are working with global variable that appear at the top of your file. You can begin writing SVG elements right after your list of global variables, and set up the outer layer of elements, the SVG root node and a <g> element that we will use for grouping and scaling the whole SVG image to be visible in a web browser. Your code should be structured like this:

declare default element namespace "http://www.w3.org/2000/svg"; 
declare namespace tei="http://www.tei-c.org/ns/1.0";
declare variable $mitfordColl := collection ('/db/mitford');
declare variable $lettersColl := collection('/db/mitford/letters');
declare variable $letterFiles := $lettersColl/*;
declare variable $countLetterFiles := count($letterFiles);
declare variable $letterDates := $lettersColl//tei:teiHeader//tei:msDesc//tei:head/tei:date/@when;
declare variable $letterYears := $letterDates/tokenize(string(), '-')[1];
<!--ebb: MORE GLOBAL VARIABLES To DEFINE MEASUREMENTS AND VALUES AS YOU KEEP WORKING-->
<svg>
   <g>
      <line x1="??" y1="??" x2="??" y2="??" style="??;??;"/>  
      
         {
         <!--ebb: FLWOR statements will go here, inside a pair of curly braces-->
         }
      
   </g>
   
</svg>

Notice that unlike FLWOR statements, global variables are always followed by a semicolon ;. That detail makes it possible for us to wrap several lines of code and nested FLWOR statements inside a global variable, much like we will do at the very end of this exercise.

Think about how to plot a vertical line in SVG, and which variables you have defined that will help you plot the start of the line at y1 and the end of it at y2. Remember to use curly braces { } to activate an XQuery variable to fill SVG attributes values.

Viewing your output

You should be able to plot the timeline now! Run your results with the Eval button, and view them as XML Output in the results window to look at your code. You should see SVG generated with its namespace in the root node, and your should see a simple SVG file containing a line element. You can view the SVG as a graphic in XQuery by toggling the XML Output option to Direct Output, but you probably will not see your entire line. That is because we need to set the width and height attributes on our SVG and set up the long vertical line to be viewable in a browser window on scrolling down.

While you can view your SVG as Direct Output in eXide, you may wish to see it in a browser window for a better view. Later on in this exercise (and in class), we will show you how to set up your query to save an SVG file into the eXist database, and access it from a web browser. But for the moment, you can copy your svg output in your Results window into a new SVG file that you open in <oXygen/>, save your file with a .svg file extension, and open it locally in a web browser.

Setting the viewport and shifting things with transform="translate(x, y)" so you can see the full line:

To make your long line visible, you want to estimate something wider than its widest x value and something a little longer than its largest y value so that you program a viewable area for your SVG. When generating SVG with calculated values as we are doing, this can be tricky, so we usually output our code first and read its maximum values before plugging in what is known as the viewport. To create a viewport, you need to add @height and @width attributes to your <svg> element. We did this in our SVG timeline by using raw numbers without units, estimating a bit beyond our largest y value and our widest x value, thinking about how wide we will eventually want to make our file.

We also decided to shift our SVG over a little bit so that if we use 0, 0 coordinates, the timeline won't be flush against the top and side of the screen. To do this, we work with the <g> element, which bundles the SVG elements into a group. Within the viewport we have defined on the <svg> element, we shift the <g> to adjust the x and y values of the plotted elements inside over by x units and up or down by y units. Here is how we did it, but you may decide you would like to position your plot a little differently:

  <svg width="500" height="3000">
   <g transform="translate(30, 100)">
          <line x1="??" y1="??" x2="??" y2="??" style="??;??;"/>  
          
         {
         <!--ebb: FLWOR statements will go here, inside a pair of curly braces-->
         }      
   </g> 
</svg>      
      

You have lots of options for scaling here, and we encourage you to experiment with various ways to shrink, expand, rotate, or skew your timeline. Here are some excellent resources on the viewbox and transform and scaling properties in SVG:

• Understanding SVG coordinate systems and transformations: the viewport, viewBox, and @preserveAspectRatio
• Understanding SVG coordinate systems and transformations: the @transform attribute (translating, rotating, and more)

After browsing these pages, see if you can shrink your timeline a little or alter its angle on the screen!

Marking the years on the timeline

We want to see years marked on our timeline, so we need to mark these at those regular 365 day intervals aross our line. You could do this by hand, but there is a better way that we will show you here. To set a series of marks along a line at regular intervals, you need to break a number into regular units. This is a special application of the for loop, to generate a series of integers within the span of years represented in our timeline. In your global variables, you should have a calculation of the number of years in your timeline, and this is what we want to work with to create regular hashmarks for each year. The syntax for the for loop that breaks the number 10 down into the integers that lead up to it (0, 1, 2, 3, etc up to 10), is this:

for $i in (0 to 10)

• To create hashmarks, you want to work with that range variable so that on each cycle of the loop, it will plot a new regularly spaced line from the zero starting point of the line to the end point. (You only need to draw one line in the return of your for loop for XQuery to work its magic.) You know that the dates are spaced apart by a factor of 365, so you will want to multiply $i by that factor as well to generate y positions for each hash mark.
• Define a variable inside your for loop so that you can output the text of the date next to your hash mark. Hint: You can do this by adding the minimum year for the collection to $i.
• Keep in mind that your hashmarks are horizontal lines, which means that you have one "y" value, but an "x" value that moves from left to right across the screen.
• You want to label your hashmarks, so you will want to return an SVG <line> element together with a text element positioned nearby in a legible spot. You will notice that XQuery generates an error when you try to output multiple SVG elements in a return statement! That is because XQuery requires you to output just one thing in a FLWOR return, and you are giving it several things to output. With SVG you deal with this by wrapping all the elements together in an SVG <g> element (which literally creates a single group of elements). Any positioning values you set in the root <SVG> or ancestor <g> elements will be inherited by this new <g> you place here.
• You want to experiment a bit to figure out distances that make sense and look good to you for plotting hashmarks and text labels. (How many pixels to the left or right of your hashmark should the text sit?)

Plotting counts of letters for each year

Our last SVG drawing task is to plot circles of varying sizes on or near our hashmarks, so we can see at a glance which years feature the most letters in our collection. Your output doesn’t need to look exactly like ours , but we give this to you as an idea. You can choose the plot the relative quantities of letters in other ways, as you wish, perhaps with rectangles or other SVG shapes, and perhaps centered on the line or set off to the right or left. However you choose to plot this, you should also output a text label to indicate the actual number of letters in the collection written in a given year.

You should have created a variable in the previous step that would automatically generate the four digit year values to label each hashmark. Work with that variable now, together with a helpful global variable we defined for you at the start of this assignment, so that you:

• Define a variable that looks up the letters whose date matches the four-digit date at each instance of $i, This should locate all of the letters in a given year.
• Define another variable holding the count() of these letters. We will need this to plot our circles (or rectangles or other shapes as you wish)
• Add to your return a new SVG element to be output with each iteration of $i in the for loop, and experiment with setting its dimensions, its fill, and style. Add an SVG text element to label each of these with a literal count of the letters in the year indicated at $i.

Use Eval to run your code and view your SVG. Copy your XQuery into a text file, save it following our standard homework naming conventions, and upload it to Courseweb for this exercise. However, you may wish to save your query as well as your output in eXist to view it in the browser.

Saving your XQuery and your Output files in eXist, and accessing them on the web

If you are one of my enrolled students and I have given you write access to the database, you will be able to log in to the database with a username and password I have provided you. Once you are logged in you will be able to create a new directory and save your input scripts in the eXide window. You will also be able to write your scripts so that they create an output file that saves inside the database. In the eXide workspace window, go to File and Manage, and create a folder for yourself (with one word, no spaces) using part of your name or userid.

To save your output file in your new directory, you will need to do define one more global variable, the biggest of them all: a variable that contains the entire SVG document you coded within your eXist script. You will declare a variable whose value equals the entire contents of the <svg> element, and you can do that with the handy semicolon that always concludes a global variable in XQuery:

xquery version "3.0";
declare default element namespace "http://www.w3.org/2000/svg"; 
declare namespace tei="http://www.tei-c.org/ns/1.0";
declare variable $mitfordColl := collection ('/db/mitford');
declare variable $lettersColl := collection('/db/mitford/letters');
declare variable $letterFiles := $lettersColl/*;
declare variable $countLetterFiles := count($letterFiles);
declare variable $letterDates := $lettersColl//tei:teiHeader//tei:msDesc//tei:head/tei:date/@when;
declare variable $letterYears := $letterDates/tokenize(string(), '-')[1];
<!--ebb: MORE GLOBAL VARIABLES. . .-->

declare variable $ThisFileContent := 
<svg width="500" height="3000">
   <g transform="translate(30, 100)">
          <line x1="??" y1="??" x2="??" y2="??" style="??;??;"/>  
          
         {
         <!--ebb: FLWOR statements here, inside a pair of curly braces-->
         }      
   </g> 
</svg> ;

let $filename := "timeline.svg"
let $doc-db-uri := xmldb:store("/db/yourFolder", $filename, $ThisFileContent)
return $doc-db-uri
(: Output at http://newtfire.org:8338/exist/rest/db/yourFolder/timeline.svg :)     
      

This works by creating a variable that actually stores an entire output file, and then, with one last FLWOR, encodes it with a special function, xmldb:store(), which takes three arguments to give it a filepath in eXist, a filename that you can define, and finally the file content that you encoded in SVG.

That is more than enough for now! Copy and paste your XQuery code into a text file, name it according to our standard homework conventions, and upload it to Courseweb. In part 2 of this assignment we will add information to this timeline to plot the duration of a correspondence between Mitford and one of her friends in our collection.