|
|
| (5 intermediate revisions by 2 users not shown) |
| Line 1: |
Line 1: |
| − | Editing the expo website is an adventure. Until now, there was no guide which explains the whole thing as a functioning system. Learning it by trial and error is non-trivial.
| + | ==This page has now moved== |
| | | | |
| − | The website needs improvement, perhaps a complete overhaul. However, it is impossible to go about fixing it properly until we know how the whole thing works.
| + | It's now part of the [http://expo.survex.com/handbook/update.htm expo handbook]. Please link/update that page henceforth. |
| | | | |
| − | This manual is organized in a how-to sort of style. The categories, rather than referring to specific elements of the website, refer to processes that a maintainer would want to do.
| + | A few remnants have been left here until they are properly merged. |
| | | | |
| − | ==How to update things on expo.survex.com==
| + | ----------- |
| | | | |
| − | ===Getting a username and password===
| + | To edit the website, you need a mercurial client. If you are using Windows, [http://tortoisehg.bitbucket.org/ TortoiseHg] is highly recommended. Lots of tools for Linux and mac exist too [http://mercurial.selenic.com/wiki/OtherTools ], both GUI and command-line: |
| | | | |
| − | Use these credentials for access to the site:
| |
| − | The user is 'expo', with a beery password. Ask someone if this isn't enough clue for you.
| |
| | | | |
| − | ===The repositories=== | + | ===Using Mercurial/TortoiseHg in Windows=== |
| | | | |
| − | All the expo data is contained in 4 'mercurial' repositories, and a big bucket of plain files at expo.survex.com. This is currently hosted on Julian Todd's server ('seagrass'). Mercurial* is a distributed version control system which allows collaborative editing and keeps track of all changes so we can roll back and have branches if needed.
| + | In Windows: install Mercurial and TortoiseHg of the relevant flavour from http://mercurial.selenic.com/downloads/ (ignoring antivirus/Windows warnings). |
| − | | |
| − | The site has been split into four parts:
| |
| − | * '''expoweb''' - the website itself, including generation scripts
| |
| − | * '''loser''' - the survex survey data
| |
| − | * '''tunneldata''' - the tunnel data and drawings
| |
| − | * '''troggle''' - the Troggle expo info viewer thingy
| |
| − | | |
| − | All the scans, photos and videos have been removed from version-control and are just files. See below for details on that.
| |
| − | | |
| − | ===How the website works===
| |
| − | | |
| − | Part of the website is static HTML, but quite a lot is generated by scripts. So anything you check in which affects cave data or descriptions won't appear on the site until the website update scripts are run. This happens automatically every 15 mins (on the hour, quarter past etc). You can also kick of a manual update: see 'The expoweb-update script' below for details.
| |
| − | | |
| − | Increasingly much of the site is generated by troggle. That will reflect updated info immediately if it has the data, but it only reads in all the expo files when poked. When I find out how to do this I'll write it down here...
| |
| − | | |
| − | ===Quick start===
| |
| − | | |
| − | If you know what you are doing here is the basic info on what's where:
| |
| − | | |
| − | '''expoweb''' on seagrass (The Website)
| |
| − | hg [clone|pull|push] ssh://expo@seagrass.goatchurch.org.uk/expoweb
| |
| − | | |
| − | '''loser''' on seagrass (The survey data)
| |
| − | hg [clone|pull|push] ssh://expo@seagrass.goatchurch.org.uk/loser
| |
| − | | |
| − | '''tunneldata''' on seagrass (The Tunnel drawings)
| |
| − | hg [clone|pull|push] ssh://expo@seagrass.goatchurch.org.uk/tunneldata
| |
| − | | |
| − | '''troggle''' on seagrass (The data-driven part of the website)
| |
| − | hg [clone|pull|push] ssh://expo@seagrass.goatchurch.org.uk/troggle
| |
| − | | |
| − | | |
| − | Photos, scans (logbooks, drawn-up cave segments)
| |
| − | (This is about 30GB of stuff which you probably don't actually need locally)
| |
| − | To sync the files from seagrass to local expoimages directory:
| |
| − | rsync -av expo@seagrass.goatchurch.org.uk:expoimages /home/expo/fromserver
| |
| − | | |
| − | To sync the local expoimage directory back to seagrass:
| |
| − | rsync -av /home/expo/fromserver/expoimages expo@seagrass.goatchurch.org.uk:
| |
| − | | |
| − | (do be careful not to delete piles of stuff then rsync back - as it'll all get deleted on the server too!)
| |
| − | | |
| − | ===Editing the website===
| |
| − | | |
| − | To edit the website, you need a mercurial client. If you are using Windows, [http://tortoisehg.bitbucket.org/|TortoiseHg] is highly recommended. Lots of tools for Linux and mac exist too [http://mercurial.selenic.com/wiki/OtherTools ], both GUI and command-line:
| |
| − | * For '''Ubuntu''' dummies and '''GUI''' lovers:
| |
| − | ** check [http://g0blin.co.uk/programming/installing-tortoisehg-2-0-under-ubuntu-10-10/ this how to install the latest Mercurial version] which is not in the usual repositories. In Ubuntu 11.04 you can just install mercurial and tortoisehg from synaptic, then restart nautilus ''$nautilus -q''. If it works, you'll be able to see the menus of Tortoise within your Nautilus windows.
| |
| − | ** (optional) f you also want to view your local copy of the website in the browser (in order to double check your changes before commiting) then you need to install a lamp server (including apache, php...). Simply do: ''sudo apt-get install tasksel'' and then ''sudo tasksel install lamp-server'', or either follow instructions [https://help.ubuntu.com/community/ApacheMySQLPHP| here], and all will be installed in one step. Finally, don't forget to set the correct address to your local copy in this file: ''/etc/apache2/sites-available/config/default'' (with the text editor, replace ''/var/www'' with the folder where your expo web copy is)
| |
| − | | |
| − | Once you've downloaded and installed a client, the first step is to create what is called a checkout of the website or section of the website which you want to work on. This creates a copy on your machine which you can edit to your heart's content. The command to initially check out ('clone') the entire expo website is
| |
| − | hg clone ssh://expo@seagrass.goatchurch.org.uk/expoweb
| |
| − | for subsequent updates
| |
| − | hg update
| |
| − | will generally do the trick.
| |
| − | | |
| − | In TortoiseHg, merely right-click on a folder you want to check out to, choose "Mercurial checkout," and enter
| |
| − | ssh://expo@seagrass.goatchurch.org.uk/expoweb
| |
| − | | |
| − | After you've made a change, commit it to you local copy with
| |
| − | hg commit (you can specify filenames to be specific) | |
| − |
| |
| − | or right clicking on the folder and going to commit in TortoiseSVN.
| |
| − | | |
| − | In order to commit you will need to configure a username the first time. TortoiseSVN sorts this out for you but for using command line tools you need to create/edit ~/.hgrc and add
| |
| − | [ui]
| |
| − | username=YourName<your@email.address>
| |
| − | | |
| − | This means that changes are identified as done by you, which can be useful info.
| |
| − | | |
| − | | |
| − | That has stored the changes in your local mercurial DVCS, but it has not sent anything back to the server. To do that you need to:
| |
| − | hg push
| |
| − | | |
| − | If someone else is editing the same bit at the same time you may also need to
| |
| − | hg merge
| |
| − | | |
| − | None of your changes will take effect, however, until the server checks out your changes and runs the expoweb-update script.
| |
| − | | |
| − | Mercurial is quite helpful with its comments and will generally say things like
| |
| − | 'new head, do hg update to bring in changes', or 'conflict found, do hg merge then hg commit to merge'. In general if you have no clue and do whatever the comments say things will work out just fine. If you have to do -force to make things happen then that's a good time to go find an expert.
| |
| − | | |
| − | ===The expoweb-update script===
| |
| − | | |
| − | The script at the heart of the website update mechanism is a makefile that runs the various generation scripts. It (along with an update from the repository) is run every 15 minutes as a cron job (at 0,15,30 and 45 past the hour), but if you want to force an update more quickly you can run it here: [facility to be announced when working...]
| |
| − | | |
| − | The scripts are generally under the 'noinfo' section of the site just because that has some access control.
| |
| − | | |
| − | ===Updating cave pages===
| |
| − | | |
| − | Cave description pages are automatically generated from a comma separated values (CSV) table named CAVETAB2.CSV by a perl script called make-indxal4.pl . make-indxal4.pl is called automatically.
| |
| − | | |
| − | The first step is to check out, edit, and check in CAVETAB2.CSV, which is at
| |
| − | /expoweb/noinfo/CAVETAB2.CSV
| |
| − | | |
| − | You need to be somewhat careful with the formatting; each cell needs to be only one line long (i.e. no newlines) or the script will get confused.
| |
| − | | |
| − | And then run expoweb-update as above.
| |
| − | | |
| − | ===Updating expo year pages===
| |
| − | | |
| − | Each year's expo has a documentation index which is in the folder
| |
| − | | |
| − | /expoweb/years
| |
| − | | |
| − | , so to checkout the 2011 page, for example, you would use
| |
| − | | |
| − | hg clone ssh://expo@seagrass.goatchurch.org.uk/expoweb/years/2011
| |
| − | | |
| − | ===Adding typed logbooks===
| |
| − | | |
| − | Logbooks are typed up and put under the years/nnnn/ directory as 'logbook.txt'.
| |
| − | | |
| − | The formatting is largely freeform, but a bit of markup ('===' around header, bars separating date, <place> - <description>, and who) allows the troggle import script to read it correctly. The underlines show who wrote the entry. There is also a format for time-underground info so it can be automagically tabulated.
| |
| − | | |
| − | So the format should be
| |
| − | ===2009-07-21|204 - Rigging entrance series| Becka Lawson, Emma Wilson, <u>Jess Stirrups</u>, Tony Rooke===
| |
| − | | |
| − | <Text of logbook entry>
| |
| − | | |
| − | T/U: Jess 1 hr, Emma 0.5 hr
| |
| − | | |
| − | | |
| − | ===Ticking off QMs===
| |
| − | | |
| − | To be written.
| |
| − | | |
| − | | |
| − | ===Maintaining the survey status table===
| |
| − | At [http://expo.survex.com/surveys/surtabnam.html] there is a table which has a list of all the surveys and whether or not they have been drawn up, and some other info.
| |
| − | | |
| − | This is generated by the script tablizebyname-csv.pl from the input file Surveys.csv
| |
| − | | |
| − | ===History===
| |
| − | | |
| − | The CUCC Website was originally created by Andy Waddington in the early 1990s and was hosted by Wookey. The VCS was CVS. The whole site was just static HTML, carefully designed to be RISCOS-compatible (hence the 10-character filenames) as both Wadders and Wookey were RISCOS people then. Wadders wrote a huge amount of info collecting expo history, photos, cave data etc.
| |
| − | | |
| − | Martin Green added the SURVTAB.CSV file to contain tabulated data for many caves, and a script to generate the index pages from it. Dave Loeffler added scripts and programs to generate the prospecting maps. The server moved to Mark Shinwell's machine in the early 2000s, and the VCS was updated to subversion.
| |
| − | | |
| − | After expo 2009 the VCS was updated to hg, because a DVCS makes a great deal of sense for expo (where it goes offline for a month or two and nearly all the year's edits happen).
| |
| − | | |
| − | The site was moved to Julian Todd's seagrass server, but the change from 32-bit to 64-bit machines broke the website autogeneration code, which was only fixed in early 2011, allowing the move to complete. The data/site has been split into 4 separate repositories: the website, the survey data, the tunnel data, and troggle - a django app to generate live data and statistics.
| |
| − | | |
| − | From June 2011 the whole site has been served through troggle, which simply falls back to the old static site where it does not have overlying functionality. This allows online editing of static pages and helps manage the slow transition from static to dynamic site.
| |
| | | | |
| | ==Automation on cucc.survex.com/expo== | | ==Automation on cucc.survex.com/expo== |
A few remnants have been left here until they are properly merged.
The way things normally work, python or perl scripts turn CSV input into HTML for the website. Note that:
The following are questions for people who know the expo website well, which stumped Aaron.