The making of hackersanddesigners.nl
Intro
This article gives an insight into the development of the new H&D website, describing its new and expanded functionalities. It is addressed both to readers who are already familiar with coding and anyone who wants to learn more about using Mediawiki as a infrastructure and the working process within the H&D approach to making (and breaking) things.
The new website shall reflect the diversity of the H&D activities and propose an online space for workshopping and a growing archive.
The H&D website runs on a Mediawiki installation, the same software that the free online encyclopedia Wikipedia is built on. Wiki websites function both as frontend as well as backend spaces. The backend is used for storage, uploading and editing, whereas the frontend serves and present the content in a selective manner. The wiki enables a variety of users to work and expand the wiki database, while simultaneously having access to the same resources (e.g.reading through published articles).
Providing the wiki content through a frontend website allows to browse its large amount of information in an easier and faster way than having to move through the wiki’s idiosyncratic backend navigation architecture (which is more of a search tool than a navigation bar). Still, wiki’s garden architecture is pretty interesting and enables a rethinking of web design, dominant backend systems and their imposed structures.
Hierarchy & structure
When I began to work on the new version of H&D website, I spent quite some time getting a sense of how this wiki had been shaped over the previous three years of its existence. Wikis often promote a flat hierarchy, such as linking and backlinking between pages. H&D’s wiki makes use of that approach while ignoring all those red links that are automatically created whenever a page does not yet exist. Everything requires to be linked and needs to exist as a page. This process promotes a full flat-ontology for each piece of data.
While keeping a flat hierarchy and ignoring hardcore linking and backlinking of pages was functional, H&D’s wiki was missing a straightforward way of creating new content. As part of the explorative approach taken from the very beginning, you could either make an event page or an article page. Both methods however require you to copy-paste particular syntaxes from previously created pages, which is very prone to causing errors. Nonetheless, Mediawiki offers ‘forms’ and ‘templates’ as a way to define the content structure of a page, which outlines what kind of information is required to create that page.
An immediate benefit of this is deduplicating variants of the same tag or token you are adding to fill out a field (eg. was it ‘meetup’ or ‘Meetup’ or ‘meet-up’? Wonder no more). This is not just petty grammar, the wiki engine will otherwise turn that link to ‘Meetup’ red if it’s not spelled correctly! The page is missing! And then you make yet another page about ‘Meetup’ but spelled this time as ‘meet-ups.’
Considering encyclopaedic minds and needs, this all sounds basic and banal. But, what if you are just defining and adding content while you go with it, as in H&D’s wiki case?
Manually adding all the information necessary to create the article is quite cumbersome, especially when the content is being constantly modified. My solution was to introduce built-in fields, where the editor can select metadata such as dates, names of collaborators and categories. That metadata gives articles a home and makes content creation more intuitive without being too pedantic–as H&D’s wiki is still not following in many occasions the wiki mindset on how to layout and organise information.
After three years of usage, it was the perfect time to review H&D’s wiki structure: there was enough content and trials to work with, making it possible to decide where to leave freedom and where improvements were needed.
(Here’s a tutorial for those who want to learn more about wiki editing and mark-up)
Toolkit & front-end
The frontend runs as a Python app. I never wrote Python before, but the language turned out to be part of H&D’s practice, so I just went for it. I opted to avoid frameworks, forcing myself to learn almost everything necessary to make a website from scratch (eg., starting with CGI and then embracing Python’s WSGI).
Concepts
One of the larger challenges was how to create a proper navigation menu.
The wiki lies on a flat surface, linking articles between each other. While the wiki gives access to special search/filter pages where you can look at all the pages with particular categories, templates, properties etc., there is no way to fetch that view through `json`.
As I expected, Semantic Mediawiki has an option to create collections of pages with common metadata elements. For example, find and group together all pages within certain dates-range AND with x, y, z properties but NOT with a, b, c properties.
This feature is called Concepts. It was the smartest way I found to let editors define site sections directly from the wiki, which would then be fetched as navigation links on the frontend. This avoided the need to set up hard coded navigation items and as such, keeping a fluid control of the content in the frontend from and through the wiki.
Tools
A tool that I used extensively for building the frontend was BeautifulSoup. At its core, BeautifulSoup is a web-scraper plugin. In the case of building the wiki, I exploited it as a fine-grain filter tool to clean up undesired `html` tags fetched by each `json` call. It worked magnificently, providing a very detailed level of control, and making it much easier to add atomic `css` classes to all `html` tags, thus forming a template. Mediawiki outputs so much wrapping `divs` that I felt obliged to get rid of them from the final template. I still wonder why this happens and if it is a problem of historical heritage due to maintaining a big and old framework that’s being modernised over time, or whether they just fell into the div wormhole and never got out of it.
Using Python instead of, say, Javascript, felt like a better choice for this project mostly because it helped me to dive into data structures. It was my first time making a restful API app. Although `json` exists because of Javascript, Python feels conceptually more aligned when learning programming and working with datasets. As Python suggests, use `try` and see what happens (`except`), instead of assuming that some particular data might be in there by using `if` this exist (`else`...). It is a different mindset that I appreciated and learned to adopt over the three months of the project.
Some thoughts on working with the wiki
Because of H&D’s activities and of its structure, in my view the wiki has a lot of potential in the workshopping context. When needed, a hundred new users can sign up and make use of the space to document, draft, write and edit content, all at the same time and without a fuss.
The frontend, conversely, becomes an easier to access place to check for upcoming events, as well as for browsing through the wiki content in a more focused, well presented manner. In a way, it can be seen as one of the output forms that the wiki can produce, as much as a book can do for a selection of wiki articles.
The H&D website becomes both a workshop AND a library.
Repository
→ Check the github repo for all the inner workings.