The making of hackersanddesigners.nl: Difference between revisions

From Hackers & Designers
No edit summary
No edit summary
(One intermediate revision by one other user not shown)
Line 6: Line 6:
[[File:Hd-making-of--init.gif]]
[[File:Hd-making-of--init.gif]]


This article gives an insight into the development of the new H&D website, describing its new and expanded functionalities. It is addressed both to readers who are already familiar with coding and anyone who wants to learn more about using Mediawiki as a infrastructure and the working process within the H&D approach to making (and breaking) things.  
This article shares the development of the H&D website and describes its new and expanded functionalities. It is addressed to readers already familiar with coding, as well as anyone interested in learning about using Mediawiki as an infrastructure, H&D's work process, and our approach to making and breaking things. The new website reflects the diversity of the H&D activities and proposes an online space for workshopping and a growing archive.


The new website shall reflect the diversity of the H&D activities and propose an online space for workshopping and a growing archive.
The H&D website runs on a Mediawiki installation. This is the same software on which the free online encyclopedia Wikipedia is built. Wiki websites function as both frontend and backend spaces. The frontend serves and present the content in a selective manner. The backend is used for storage, uploading, and editing. The wiki enables a variety of users to work and expand the wiki database while having simultaneous access to the same resources (e.g. reading through published articles).


The H&D website runs on a Mediawiki installation, the same software that the free online encyclopedia Wikipedia is built on. Wiki websites function both as frontend as well as backend spaces. The backend is used for storage, uploading and editing, whereas the frontend serves and present the content in a selective manner. The wiki enables a variety of users to work and expand the wiki database, while simultaneously having access to the same resources (e.g.reading through published articles).
Providing the wiki content through a frontend website allows users to easily and quickly access a large amount of information. This is in comparison to having to move through the wiki’s idiosyncratic backend navigation architecture, which is more of a search tool than a navigation bar. Still, wiki’s architecture is interesting and enables a rethinking of web design, dominant backend systems, and their imposed structures.
 
Providing the wiki content through a frontend website allows to browse its large amount of information in an easier and faster way than having to move through the wiki’s idiosyncratic backend navigation architecture (which is more of a search tool than a navigation bar). Still, wiki’s garden architecture is pretty interesting and enables a rethinking of web design, dominant backend systems and their imposed structures.


== Hierarchy & structure ==
== Hierarchy & structure ==


When I began to work on the new version of H&D website, I spent quite some time getting a sense of how this wiki had been shaped over the previous three years of its existence. Wikis often promote a flat hierarchy, such as linking and backlinking between pages. H&D’s wiki makes use of that approach while ignoring all those red links that are automatically created whenever a page does not yet exist. Everything requires to be linked and needs to exist as a page. This process promotes a full flat-ontology for each piece of data.
I spent quite some time getting a sense of how this wiki had been shaped over the previous three years of its existence when I began to work on the new version of H&D website. Wikis often promote a flat hierarchy, such as linking and backlinking between pages. H&D’s wiki makes use of that approach while ignoring all those red links that are automatically created when a page does not yet exist. Everything requires to be linked and needs to exist as a page. This process promotes a flat ontology for each piece of data.


While keeping a flat hierarchy and ignoring hardcore linking and backlinking of pages was functional, H&D’s wiki was missing a straightforward way of creating new content. As part of the explorative approach taken from the very beginning, you could either make an event page or an article page. Both methods however require you to copy-paste particular syntaxes from previously created pages, which is very prone to causing errors. Nonetheless, Mediawiki offers ‘forms’ and ‘templates’ as a way to define the content structure of a page, which outlines what kind of information is required to create that page.  
H&D’s wiki was missing a straightforward way of creating new content. As part of the explorative approach taken from the beginning, you can either make an event page or an article page. Both methods, however, require you to copy-paste particular syntaxes from previously created pages, which is very prone to causing errors. Nonetheless, Mediawiki offers ‘forms’ and ‘templates’ as a way to define the content structure of a page. This outlines what kind of information is required to create that page.  


An immediate benefit of this is deduplicating variants of the same tag or token you are adding to fill out a field (eg. was it ‘meetup’ or ‘Meetup’ or ‘meet-up’? Wonder no more). This is not just petty grammar, the wiki engine will otherwise turn that link to ‘Meetup’ red if it’s not spelled correctly! The page is missing! And then you make yet another page about ‘Meetup’ but spelled this time as ‘meet-ups.’  
An immediate benefit of this is de-duplicating variants of the same tag or token is that you are adding to fill out a field (was it ‘meetup’ or ‘Meetup’ or ‘meet-up’? Wonder no more). This is not just petty grammar. The wiki engine will otherwise turn that link to ‘Meetup’ red if it’s not spelled correctly! The page is missing! And then you make yet another page about ‘Meetup’ but spelled this time as ‘meet-ups.’  


Considering encyclopaedic minds and needs, this all sounds basic and banal. But, what if you are just defining and adding content while you go with it, as in H&D’s wiki case?  
Considering encyclopedic minds and needs, this all sounds banal. But, what if you are just defining and adding content while you go with it, as in H&D’s wiki case?  


Manually adding all the information necessary to create the article is quite cumbersome, especially when the content is being constantly modified. My solution was to introduce built-in fields, where the editor can select metadata such as dates, names of collaborators and categories. That metadata gives articles a home and makes content creation more intuitive without being too pedantic–as H&D’s wiki is still not following in many occasions the wiki mindset on how to layout and organise information.
Manually adding all the information necessary to create the article is cumbersome, especially when the content is being constantly modified. My solution was to introduce built-in fields, where the editor can select metadata such as dates, names of collaborators, and categories. That metadata gives articles a home and makes content creation more intuitive without being too pedantic. This is helpful, as H&D’s wiki often does not follow the wiki mindset on how to layout and organize information.


After three years of usage, it was the perfect time to review H&D’s wiki structure: there was enough content and trials to work with, making it possible to decide where to leave freedom and where improvements were needed.
After three years of usage, it was the perfect time to review H&D’s wiki structure. There was enough content and trials to work with, making it possible to decide where to leave freedom and where improvements were needed.


([[Wiki Tutorial|Here’s a tutorial]] for those who want to learn more about wiki editing and mark-up)
For those who want to learn more about wiki editing and mark-up, a wiki tutorial may be [[Wiki Tutorial|found here]].


== Toolkit & front-end ==
== Toolkit & front-end ==


The frontend runs as a Python app. I never wrote Python before, but the language turned out to be part of H&D’s practice, so I just went for it. I opted to avoid frameworks, forcing myself to learn almost everything necessary to make a website from scratch (eg., starting with CGI and then embracing Python’s WSGI).
The frontend runs as a Python app. I never wrote Python before, but the language turned out to be part of H&D’s practice, so I just went for it. I opted to avoid frameworks and forced myself to learn almost everything necessary to make a website from scratch. I started with server CGI and then embraced Python’s WSGI.


=== Concepts ===
=== Concepts ===


One of the larger challenges was how to create a proper navigation menu.
One of the bigger challenges was how to create a proper navigation menu.


The wiki lies on a flat surface, linking articles between each other. While the wiki gives access to special search/filter pages where you can look at all the pages with particular categories, templates, properties etc., there is no way to fetch that view through `json`.
The wiki lies on a flat surface, linking articles between each other. While the wiki gives access to special search and filter pages where you can look at all the pages with particular categories, templates, properties, etc., there is no way to fetch that view through `json.`  


As I expected, Semantic Mediawiki has an option to create collections of pages with common metadata elements. For example, find and group together all pages within certain dates-range AND with x, y, z properties but NOT with a, b, c properties.
As I expected, Semantic Mediawiki has an option to create collections of pages with common metadata elements. For example, it gave the option to find and group together all pages within a date range AND with x, y, z properties but NOT with a, b, c properties.


This feature is called Concepts. It was the smartest way I found to let editors define site sections directly from the wiki, which would then be fetched as navigation links on the frontend. This avoided the need to set up hard coded navigation items and as such, keeping a fluid control of the content in the frontend from and through the wiki.
This feature is called Concepts. It was the smartest way I found to let editors define site sections directly from the wiki. These sections would then be fetched as navigation links on the frontend. This avoided the need to set up hard-coded navigation items. As such, it keeps a fluid control of the content in the frontend from and through the wiki.


=== Tools ===
=== Tools ===


A tool that I used extensively for building the frontend was BeautifulSoup. At its core, BeautifulSoup is a web-scraper plugin. In the case of building the wiki, I exploited it as a fine-grain filter tool to clean up undesired `html` tags fetched by each `json` call. It worked magnificently, providing a very detailed level of control, and making it much easier to add atomic `css` classes to all `html` tags, thus forming a template. Mediawiki outputs so much wrapping `divs` that I felt obliged to get rid of them from the final template. I still wonder why this happens and if it is a problem of historical heritage due to maintaining a big and old framework that’s being modernised over time, or whether they just fell into the div wormhole and never got out of it.
A tool that I used extensively for building the frontend was BeautifulSoup. At its core, BeautifulSoup is a web-scraper plugin. In the case of building the wiki, I exploited it as a fine-grain filter tool to clean up undesired `html` tags fetched by each `json` call. It worked magnificently, providing a very detailed level of control. This fine control made it much easier to add atomic `css` classes to all `html` tags, thus forming a template. Because Mediawiki outputs so much wrapping `divs,` though, I felt obliged to get rid of them from the final template. I still wonder why this happens. It could be a problem of historical heritage due to maintaining a big and old framework that’s being modernised over time, or maybe Mediawiki developers just fell into the div wormhole and never got out.


Using Python instead of, say, Javascript, felt like a better choice for this project mostly because it helped me to dive into data structures. It was my first time making a restful API app. Although `json` exists because of Javascript, Python feels conceptually more aligned when learning programming and working with datasets. As Python suggests, use `try` and see what happens (`except`), instead of assuming that some particular data might be in there by using `if` this exist (`else`...). It is a different mindset that I appreciated and learned to adopt over the three months of the project.
Using Python instead of, say, Javascript, felt like a better choice for this project mostly because it helped me to dive into data structures. It was my first time making a [https://en.wikipedia.org/wiki/Representational_state_transfer restful API app]. Although `json` exists because of Javascript, Python feels conceptually more aligned when learning programming and working with datasets. As Python suggests, use `try` and see what happens (`except`) instead of assuming that some particular data might be in there by using `if` this exist (`else`...). It is a different mindset that I appreciated and learned to adopt over the three months of the project.


== Some thoughts on working with the wiki ==
== Some thoughts on working with the wiki ==


Because of H&D’s activities and of its structure, in my view the wiki has a lot of potential in the workshopping context. When needed, a hundred new users can sign up and make use of the space to document, draft, write and edit content, all at the same time and without a fuss.
Because of H&D’s activities and structure, the wiki has a lot of potential in the workshopping context. When needed, a hundred new users can sign up and make use of the space to document, draft, write, and edit content, all at the same time and without a fuss.
 
The frontend, conversely, becomes an easier to access place to check for upcoming events, as well as for browsing through the wiki content in a more focused, well presented manner. In a way, it can be seen as one of the output forms that the wiki can produce, as much as a book can do for a selection of wiki articles.  


The H&D website becomes both a workshop AND a library.
The frontend, conversely, becomes an easier place to access and check for upcoming events, as well as for browsing through the wiki content in a focused and well-presented manner. The H&D website becomes both a workshop AND a library.


== Repository ==
== Repository ==


Check the [https://github.com/hackersanddesigners/had-py github repo] for all the inner workings.
All the inner workings can be found on the [https://github.com/hackersanddesigners/had-py github repo].


[[Category:Projects]]
[[Category:Publishing]]

Revision as of 19:42, 5 January 2020

MainNavigation No

Intro

Hd-making-of--init.gif

This article shares the development of the H&D website and describes its new and expanded functionalities. It is addressed to readers already familiar with coding, as well as anyone interested in learning about using Mediawiki as an infrastructure, H&D's work process, and our approach to making and breaking things. The new website reflects the diversity of the H&D activities and proposes an online space for workshopping and a growing archive.

The H&D website runs on a Mediawiki installation. This is the same software on which the free online encyclopedia Wikipedia is built. Wiki websites function as both frontend and backend spaces. The frontend serves and present the content in a selective manner. The backend is used for storage, uploading, and editing. The wiki enables a variety of users to work and expand the wiki database while having simultaneous access to the same resources (e.g. reading through published articles).

Providing the wiki content through a frontend website allows users to easily and quickly access a large amount of information. This is in comparison to having to move through the wiki’s idiosyncratic backend navigation architecture, which is more of a search tool than a navigation bar. Still, wiki’s architecture is interesting and enables a rethinking of web design, dominant backend systems, and their imposed structures.

Hierarchy & structure

I spent quite some time getting a sense of how this wiki had been shaped over the previous three years of its existence when I began to work on the new version of H&D website. Wikis often promote a flat hierarchy, such as linking and backlinking between pages. H&D’s wiki makes use of that approach while ignoring all those red links that are automatically created when a page does not yet exist. Everything requires to be linked and needs to exist as a page. This process promotes a flat ontology for each piece of data.

H&D’s wiki was missing a straightforward way of creating new content. As part of the explorative approach taken from the beginning, you can either make an event page or an article page. Both methods, however, require you to copy-paste particular syntaxes from previously created pages, which is very prone to causing errors. Nonetheless, Mediawiki offers ‘forms’ and ‘templates’ as a way to define the content structure of a page. This outlines what kind of information is required to create that page.

An immediate benefit of this is de-duplicating variants of the same tag or token is that you are adding to fill out a field (was it ‘meetup’ or ‘Meetup’ or ‘meet-up’? Wonder no more). This is not just petty grammar. The wiki engine will otherwise turn that link to ‘Meetup’ red if it’s not spelled correctly! The page is missing! And then you make yet another page about ‘Meetup’ but spelled this time as ‘meet-ups.’

Considering encyclopedic minds and needs, this all sounds banal. But, what if you are just defining and adding content while you go with it, as in H&D’s wiki case?

Manually adding all the information necessary to create the article is cumbersome, especially when the content is being constantly modified. My solution was to introduce built-in fields, where the editor can select metadata such as dates, names of collaborators, and categories. That metadata gives articles a home and makes content creation more intuitive without being too pedantic. This is helpful, as H&D’s wiki often does not follow the wiki mindset on how to layout and organize information.

After three years of usage, it was the perfect time to review H&D’s wiki structure. There was enough content and trials to work with, making it possible to decide where to leave freedom and where improvements were needed.

For those who want to learn more about wiki editing and mark-up, a wiki tutorial may be found here.

Toolkit & front-end

The frontend runs as a Python app. I never wrote Python before, but the language turned out to be part of H&D’s practice, so I just went for it. I opted to avoid frameworks and forced myself to learn almost everything necessary to make a website from scratch. I started with server CGI and then embraced Python’s WSGI.

Concepts

One of the bigger challenges was how to create a proper navigation menu.

The wiki lies on a flat surface, linking articles between each other. While the wiki gives access to special search and filter pages where you can look at all the pages with particular categories, templates, properties, etc., there is no way to fetch that view through `json.`

As I expected, Semantic Mediawiki has an option to create collections of pages with common metadata elements. For example, it gave the option to find and group together all pages within a date range AND with x, y, z properties but NOT with a, b, c properties.

This feature is called Concepts. It was the smartest way I found to let editors define site sections directly from the wiki. These sections would then be fetched as navigation links on the frontend. This avoided the need to set up hard-coded navigation items. As such, it keeps a fluid control of the content in the frontend from and through the wiki.

Tools

A tool that I used extensively for building the frontend was BeautifulSoup. At its core, BeautifulSoup is a web-scraper plugin. In the case of building the wiki, I exploited it as a fine-grain filter tool to clean up undesired `html` tags fetched by each `json` call. It worked magnificently, providing a very detailed level of control. This fine control made it much easier to add atomic `css` classes to all `html` tags, thus forming a template. Because Mediawiki outputs so much wrapping `divs,` though, I felt obliged to get rid of them from the final template. I still wonder why this happens. It could be a problem of historical heritage due to maintaining a big and old framework that’s being modernised over time, or maybe Mediawiki developers just fell into the div wormhole and never got out.

Using Python instead of, say, Javascript, felt like a better choice for this project mostly because it helped me to dive into data structures. It was my first time making a restful API app. Although `json` exists because of Javascript, Python feels conceptually more aligned when learning programming and working with datasets. As Python suggests, use `try` and see what happens (`except`) instead of assuming that some particular data might be in there by using `if` this exist (`else`...). It is a different mindset that I appreciated and learned to adopt over the three months of the project.

Some thoughts on working with the wiki

Because of H&D’s activities and structure, the wiki has a lot of potential in the workshopping context. When needed, a hundred new users can sign up and make use of the space to document, draft, write, and edit content, all at the same time and without a fuss.

The frontend, conversely, becomes an easier place to access and check for upcoming events, as well as for browsing through the wiki content in a focused and well-presented manner. The H&D website becomes both a workshop AND a library.

Repository

→ All the inner workings can be found on the github repo.