Tuesday 19 April was our ninth Hackathon of the year, and our final one for a little while as we are putting our Hackathons on hold temporarily due to our Siyavula work commitments. We had a strong turnout of 29 volunteers, and as always it was great to see the familiar faces of everyone as they arrived.

Objectives

The objectives of our Hackathons has always been to improve FHSST. The focus of the Hackathons in the first semester was to upload maths and physical science exemplar papers for Grades 10 – 12 to FullMarks. The goal here was to uploaded 180 questions and solutions per Hackathon, which averages out to around 10 questions per volunteer with a laptop. Once volunteers had mastered FullMarks we would move them on to editing chapters on Connexions.

Approach

Through our Hackathons on UCT campus we are providing a space, process and structure for volunteers to make a meaningful contribution to an education initiative, which not only provides them with a worthwhile outreach outlet, but also helps us improve FHSST. Through our project we are channelling the efforts of many individuals that are willing and happy to volunteer their time to support education, and with our focus of getting the books onto the government approved list this year, we can really add value to their contributions by maximising the impact nationally. Making a difference to education is certainly easier to do in collaboration as a team, than attempts made in isolation by individuals.

Bottlenecks, challenges and refining our process

Over the course of nine Hackathons we have smoothed out the creases and reduced the bottlenecks we experienced in our first few Hackathons. Our process now is by no means perfect, but we have certainly learned a lot along the way, and after each Hackathon we have revised and revisited the previous night’s experiences, in order to improve and become more efficient for every coming week.

Bottlenecks and challenges

Our first hackathon served as an introduction to FHSST and Siyavula, and that ran smoothly as Neels gave a presentation. Following that everyone listened and asked questions about the books and what they would be doing at the Hackathons. The second Hackathon is where things got interesting. We had 33 volunteers arrive, which was fantastic for us – this was the biggest Hackathon turnout we had seen in any of our Hackathons from 2010. While we felt we had prepared adequately for the evening, we experienced a few technical difficulties. We had two flashsticks with us that had OpenOffice as well as the exemplar papers on them, but when faced with approximately 20 people with laptops (we weren’t expecting so many brand new people), all needing OpenOffice, this was not nearly enough. In addition to having the exemplar papers on the flashsticks we had also placed the content on UCT’s student portal Vula. But, as we were relatively new to Vula, we uploaded it to an area of the website that wasn’t ideal. One of our volunteers stepped up and reuploaded the content to a more user friendly area. We also had some issues with Macs that people had, as they wouldn’t read our flashsticks, plus the copy of OpenOffice we had was the wrong version for Macs. Once again a volunteer assisted and downloaded the right version and they were able to get going. As we encouraged volunteers to use Firefox at our Hackathons, many people didn’t know the settings for UCT to get online, and our team not being UCT based, didn’t either, so there were some problems there.

In addition to these teething problems, we didn’t anticipate so many people struggling with FullMarks. In retrospect we realised that the help sheets we provided everyone with were too text heavy, and so the volunteers weren’t reading them. Our team also could have done with a few practice sessions, as not all of us had used it recently. FullMarks on its own is giving us some issues – there seem to be some bugs that are becoming more apparent the more we use it. In addition to this, the online editor is difficult to use, but that is being upgraded and the various bugs investigated, so those problems should fall away in the not too distant future.

Although we had some issues initially, the volunteers were great at stepping up and helping each other, offering assistance to us with regards to using Vula, downloading OpenOffice on our behalf (we can’t access the UCT wireless network as we don’t have logins) and helping each other get online. Despite us feeling a bit frazzled at the end of the evening, we got some positive feedback from volunteers who felt the Hackathon went well! There was certainly a great vibe in the room, and people were chatting – I think the fact that the volunteers needed each other’s help meant that they had to interact and so got to know each other, which was a definite plus of the whole experience!

Brainstorming and streamlining

After this Hackathon we had a long debriefing and brainstorming session and started planning for our next Hackathon (Hackathon 3). Hackathon 3 turned out to be our best one up to that point (taking 2010′s Hackathons into account too). We revised our process sheet and combined it with the assignment sheet, and gave clearer and easy to read instructions, with specific reference to tagging (without proper tagging the questions will not be found by teachers using FullMarks). We burned multiple copies of dvds with all the open programmes, exemplar papers and help sheets. We also changed how we would set up the room – we had a check-in station at the entrance to the venue, where on arrival volunteers would check in with me, and then proceed on to Natalia, Carine or Heather for assignments depending on their choice of subject and whether or not they had a laptop. We also arranged the desks into 3 big work stations, encouraging people to sit together and not group around small tables.

Refining our process

Prior to Hackathon 3 we had a team training session on using FullMarks, to refresh our memories and ensure we could better cope with helping the volunteers. We also had a meeting shortly before we left for the Hackathon, so that each team member was up to speed and knew where to find all of the content. This advanced and thorough planning made for a less stressful evening for our team. The volunteers still needed a fair amount of help at the Hackathon, as the process of using the FullMarks template and uploading to FullMarks was brand new to them. We found that despite our revised process document, people still weren’t reading them properly, and so didn’t write their names or FullMarks user ID’s on their assignment sheets – information we needed when checking what content had been uploaded by each volunteer.

Throughout the Hackathons we stuck with our streamlined process of arranging the room before each Hackathon, and having our registration and assignment tables at the entrance, as we felt it worked well and it meant that there wasn’t a crowd around one table all needing Heather’s attention, which is what we had to deal with last year. Over the course of the next few Hackathons we revised our process / assignment sheet , trying to take out as much text as possible, and have a diagram that at a glance would show the entire upload process in as simple a way as possible. We also made a point of sitting with a volunteer that needed help, until they had uploaded one question in full. This made a difference as it meant volunteers learned the process in full, and were better able to help not only themselves, but others at their table too.

Over the course of the Hackathons we soon realised we needed a way to better manage brand new volunteers that were bringing laptops. We had reached the stage where the majority of the laptop users didn’t need as much help anymore, but at each Hackathon we had a couple of new volunteers, and it didn’t make sense to have each of us going through the same process with new people repeatedly. It was decided that new people would be sent to a separate table where Heather would talk them through the process in full, before they could go and join a large table. From the rest of the team’s point of view this made a huge difference to the help that was needed, but it did mean that Heather spent the night repeating herself as volunteers trickled in at different times, so she couldn’t give the demonstration once or twice, but had to multiple times. As a result of this she was also quite disconnected from what happened at that particular Hackathon.

Assignment sheets and checking uploaded questions

From the start, Heather had the task of creating our assignment sheets, which involved creating a database containing all the exemplar papers and question numbers, and then creating a form letter using form fields. This meant that we ended up with assignment sheets with the assignments automatically generated and assigned to each page. Our assignment sheet was double sided, with the front page covering the upload and tagging process, and the other side giving the assignment. We split exemplar papers up into multiple assignments, with each volunteer being given an assignment of around four to five questions long, which could be broken down into smaller questions depending on the type of question given. We thought this was a reasonable length, as we were hoping for up to ten questions being uploaded per person at each Hackathon. However, over the course of Hackathon 3 to 6, volunteers were still needing a lot of help from us and so the going was slow. We also ran a focus group (which can be read about here) at Hackathon 5, which involved some of our strongest uploaders, which affected our productivity for that week.

In preparation of Hackathon 7 we decided to try an alternate method for exemplar assignments: make them shorter! Each week of Hackathons we were averaging around 40 – 50 questions, which was a much slower rate of upload than we had anticipated. In an attempt to encourage people to speed up, we cut each assignment down to 2 questions long. This meant that volunteers could feel a sense of achievement on completion of an assignment, but we could also monitor who was coming up for their second assignment. We think that this potentially made a difference – some people did move onto a second assignment, but at the same time that we implemented this, we also moved our top uploaders onto Connexions, which meant our upload count to FullMarks was affected by this change. In future Hackathons we will most likely keep assignments shorter, as it does make for a less daunting assignment.

By Hackathon 6 we decided it was time to introduce a new activity: checking of questions uploaded to FullMarks. We randomly assigned FullMarks question ID’s to volunteers, along with a step by step process of what needed to be checked for in each question, i.e. what the perfect question should look like. They needed to give constructive criticism in the comments field under each question, so that the author of that question could then make the suggested changes (the author receives an email when a question of theirs is commented on). This was not too well received – people that were present felt that they were being criticised for their work by their peers, while those that were not present received emails with comments and they didn’t know why. This was an oversight on our part, and the following day I emailed all our FHSST members to explain the purpose of the activity, that its aim was to improve the quality of the questions on FullMarks, and not intended as a criticism of them and their work.

Paper people

The people we have working on paper are an ongoing challenge for us. Ideally we would like everyone working on a computer, but not everyone has a laptop and we are struggling to organise a computer lab on campus (we did a survey amongst our volunteers and found they would prefer Hackathons to stay in a venue on campus, so we can’t move to a nearby school’s computer lab). Paper people have been making good contributions to our textbooks, as they have been working on the model solutions for the exercises in the textbooks. The only problem however, is managing this process. Volunteers haven’t always numbered the assignments correctly, or put their names on them, so when it comes to sorting through the assignments each Wednesday after a Hackathon, it presents some challenges! It also means that at some stage someone is going to have to type up all the work, and check it as they go along.

Team at tables

From Hackathon 6 we decided that at each Hackathon we were to each sit at a table with the volunteers, and work along side them. The aim of this was to get to know them better, and vice versa. At this Hackathon it ended up only being Carine sharing a table, as the rest of our laptops were given out to volunteers to use, plus there were still people needing assistance. Carine felt that she got good insight into what it is like being a volunteer at our Hackathons, as she experienced the process of using the OpenOffice template and uploading to FullMarks, and asking the table for help when she got stuck with the Math type. She became part of the friendly banter, and saw how the volunteers helped each other and problem solved together. Carine has continued to sit at a table at Hackathons, and continues to ask questions and find out about the whole Hackathon experience.

Snack table, pizza and logistics

In all our Hackathons we have always done well on the catering and logistics side of things. We always come well equipped with many power cables and multiplugs, a projector should we wish to show the volunteers anything, and all the necessary signage. Before each Hackathon we go shopping at Checkers on Main Road Rondebosch (we almost own shares in it now), and stock up on fizzy drinks, juice, water, chips, sweets, plastic cups, paper plates and serviettes. We arrive at UCT’s RW James Room L by 5pm, and set up the room and put up posters in preparation of the arrival of our volunteers. I always prepare my pizza order form early in the day, based on the Eventbrite signups and which are the most popular Butler’s pizzas. I place my order with Butler’s at 17h30, for a 19h30 delivery. They give us a 33% discount which is fantastic, and the pizza always arrives on time and piping hot.

We have done a few pizza experiments over the weeks: last year everyone was allowed to choose their own medium sized pizza, but we felt that this didn’t encourage conversation and building community. This year we created a communal pizza table, in the hopes of bringing in a slightly more social aspect to the Hackathons. Unfortunately as sometimes happens, people started taking advantage of the free pizza and piled their plates high with food and stashing it to take home, while others were more restrained and only took two or three pieces to start with. Those that held back initially ended up missing out, as by the time they were ready for their second helping, the pizza was mostly finished. As a result of this we tried a new approach: we would allocate a certain number of pizzas to each table, which meant we could control how much pizza each table received, and anyone taking more than their due would have to deal with a table of disapproving looks. This approach received mixed responses. Some were happy with it and thought it worked well, while others didn’t like it as they said they didn’t have a choice over what pizza slices they could choose. We ended up setting up a poll on Vula with three pizza options: 1) choose your own medium pizza, 2) communal pizza, or 3) pizza at each table. We had a tie between choosing your own pizza and communal pizza, so for our last Hackathon we went with communal pizza. Should we decide to do individual pizzas again, we need to warn attendees in advance so that they know to get their orders in on time, otherwise people arrive late and then don’t get pizza.

In addition to some people taking too much pizza, it has become apparent that some come along just for the food. This number is definitely in the minority, but it is a pity as the majority of our volunteers arrive and work hard, giving up three hours of their Tuesday evening each week to improve FHSST, while others come and make a very poor effort, arrive just in time for the pizza delivery, and leave shortly after eating. We will address this problem before we start Hackathons again, as aside from being unfair to those that work hard, we are not there to sponsor people’s supper with no effort on their part.

Achievements

Overall we feel that a community is being built around FHSST and that people are taking ownership of the project. We have seen volunteers returning week after week, making friends within the group, offering assistance to their peers, putting in many hours and having a great attitude towards the project. People are also learning new skills – using OpenOffice and how styles work, as well as learning about open tools that exist online. We hope that they will share their acquired knowledge with other people, especially any teachers they may know.

Conclusion

As a team we have learned how to improve our Hackathons, and the planning that is required to ensure they run smoothly. We still have a long way to go as there are always improvements that can be made, but we have an online management tool in the pipeline that once built will be able to manage Hackathons for us, based on all that we have learned over the last nine weeks and last year. We have learned about our own strengths and weaknesses, what each team member brings to the table and the important role that each of us plays within Siyavula. We have learned how to function as a team, how to rely on each other, but also how to stand alone and be independent from each other.

We have also learned a lot about volunteers and how people react in different situations. We have been so impressed with how people have stepped up and offered assistance to their fellow volunteers and to us when help was needed. We really appreciate this, especially knowing that we could rely on volunteers each week.

Overall the last nine weeks have been very insightful and we have learned so much about running Hackathons and working with volunteers. Our process is by no means perfect, and we will continue to learn and experiment with new processes once we start our Hackathons again.

 

read original post on bridget's Site