What a weekend! We just wrapped up WordCamp Ottawa 2016 at Carleton University here in Ottawa, Canada. We had approximately 160 attendees show up for the weekend, with one day full of full-length (45 min) and lightning (20 min) talks, and a second day of hands-on workshops and “un-conference” activities.
One thing we tried to do differently this year was to improve the feedback that our wonderful speakers received about the presentations they spent so much time preparing for our camp. Having attended many conferences in Canada and the United States, I’ve seen this done in various ways.
- Send out a survey to all attendees by e-mail the at the end of the conference asking them to provide their feedback on the talks they saw, as well as the overall event.
- Use an online speaker feedback service that requires attendees to create an account and login to provide their comments
- Hand out paper forms at the event
- Do nothing (it amazes me that some conferences don’t seem to focus on session feedback)
We took the analog approach. That’s right, we asked all the attendees to fill out a short paper survey after every talk.
The result? We saw a significantly higher level of engagement than by asking them to fill out a survey on the web, or by logging into a conference feedback system that requires an account.
David Bird speaking about Google Analytics at WordCamp Ottawa 2016.
Photo courtesy of Kiera Howe
Our Goal
In doing this, we had two goals:
- Collect as much feedback as possible, as quickly as possible. By collecting feedback immediately, the attendee’s thoughts are fresh.
- Get the feedback to the presenter as quickly as possible. Presenting at a conference can be nerve-wracking. We’re curious creatures, we want to know how our work was received.
How We Did It
The Form
In order to collect this feedback, we had to create a form for the attendees to complete. I “borrowed” (with permission) the template used by the ConFoo Web Developer’s Conference in Montreal (thanks Anna!) and adjusted it for our use. The idea is to keep it simple. What did the attendee think of the speaker? the subject matter? What did they enjoy? What did they feel could be improved?
This form was made so that we could fit two per 8.5″ x 11″ piece of paper. We had our printer run off a pile of copies of these, then cut all the pages in half. (This process uses a lot of paper, there was no need to waste a full page on every form!)
Distribution and Collection
We couldn’t have done this without the help of our amazing volunteers. We asked the room host to hand out a blank form to each attendee as they walked in the door, and to collect the forms as the attendees left at the end of each talk. Doing this had a few advantages:
- If we had of simply put a few feedback forms in a bag handed to the attendees at registration, they’d get crumpled and forgotten.
- Having the host at the door collecting the forms provides encouragement for the attendee to complete and hand in their forms (a little guilt never hurt anyone, right?)
The room host would then drop off all the forms to an organizer, or an organizer would go to each room and collect the forms.
Scanning & Distribution to the Speakers
This is how we took care of our 2nd goal, to get the feedback to the speakers as soon as possible. As soon as the forms had been collected, they’d be brought up to the Organizer/Speaker lounge to be scanned and e-mailed to the speaker.
To do this, I used my Neat Receipts scanner and software. It allowed me to put a stack of feedback forms into the scanner (I was easily doing 20 at a time) and turn them into a PDF that I could then e-mail.
The result? We were able to provide feedback to the speakers in under an hour after they walked off the stage.
The Result
In a conference with about 160 attendees, we collected 476 feedback forms! Keep in mind that half of the 2nd day was an unconference in which we didn’t ask for feedback forms. A few more details on the feedback received:
- 352 of these forms were for the 16 main talks on Saturday,
- 71 were for the 10 lightning talks on Saturday,
- 53 were for the 3 workshops that took place on Sunday.
The talks received between 4 and 43 feedback forms each. They averaged 16 completed forms per talk, or 22 if you only look at the 16 main talks on Saturday.
How the data will be used
A few of our presenters asked how we were going to use the data that we collected? Honestly, not as much as you’d think. We primarily did this as a bonus service to the speakers so that they can improve their presentations. We will however, look over the feedback forms for any huge red flags (these are rare) or amazing compliments (not as rare!) that could help us in next year’s speaker selection process. We also look at the comments for feedback regarding the conference in general (the room was too hot/cold, the microphone was too loud/quiet, etc.). The subject/speaker ratings, at least for me, will largely be ignored by the committee.
Reception
I think our experiment was a hit. Here’s what two of our speakers posted to Twitter about the feedback process:
https://twitter.com/Eraserheader/status/744295133236072454
..and @wpottawa organizers rocked it by sending us scans of the #wcott feedback just a few minutes after our talks!
Take note, WordCamps!— Michal Bluma (@isotrope) June 20, 2016
Call to Action
I think this system worked out rather well for the camp organizers, the speakers, and the attendees. I would love to see more camps in our community take this approach. So, WordCamp Organizing Teams…. what are you waiting for? If you want to do this at your event, and have any questions, please ask!
I love analog, and would love to see more WordCamps try this approach. Now there’s a working model, thank you!
I think this is great for speakers, specifically, because of the high response rate and immediacy. For the overall camp, I think an online survey is still the way to go. 🙂
Getting 50+ responses from a 350+ person camp, for example, is a small response rate, but still enough to get a feel for “trends” around different topics.
Thanks for the comment Andy. I totally agree. We still do the overall survey that goes out by email after the camp to catch the feedback about the overall schedule, the venue, the meal, etc.
I believe the method in this post gives a stronger result for the session feedback.
Collecting useful feedback is really important for the improvement of WordPress events. I support this 100%, and I think we can improve upon your model.
A couple of years ago I proposed creating a WordCamp app that provides schedules and feedback functions. I think the next iteration of your idea is exactly this app. By moving the feedback process to the mobile environment, we can provide a better user experience and collect more manageable data. It can also be centralized so all the data is available for WordCamp Central etc.
Had a similar discussion to this on the weekend. Would be interesting to see this integration done as part of the CampTix system used to run the ticket sales for WordCamps, since every attendee is already in that database. It would also make a really interesting example of how to use the WordPress REST API to push & pull data between a WordPress site and a native mobile app.
As a followup here, our recent iteration of this idea was removing the scanning process completely – we simply collected each speaker’s forms and delivered an envelope of feedback within 30 minutes after each talk. I personally feel this was the best of all times we’ve done this. I’m a digital girl, but in this instance physical wins.
I present quite frequently at conferences. Most of the time, the feedback is via word of mouth.
I think having such a system is great. We’re in an interesting time where some people kind of don’t want to add another app to their phone, or take an online survey, but if given an immediate chance, they’ll fill out a short form. The key is… keeping it short, but giving them enough room to provide feedback, which did well in your case.
However, I’ve also been on the organization team end for WordCamps. It creates work, so while I love this polling for feedback, you’d have to establish early on in planning the event on who will be heading such a task and follow through in a timely manner.
Thanks for the feedback Nile. With regards to time commitment, on the camp days, I’m guessing this took 50-60% of my time between going between rooms to collect feedback, scanning, e-mailing, making sure rooms had enough blank forms, etc.
A few more pieces of advice. At ConFoo, we store the scanned copies in a Dropbox, which we then share with the advisors in charge of selecting the talks for next year. This way we know which speakers/topics are good or not. To speed up the process of sending them out, we have a script that will send everything from the “pending” folder by matching file names (titles) to speaker e-mails from a spreadsheet – which we prepare in advance.
[…] By the time I got home and walked the dog, I had my feedback in my email inbox (Shawn Hooper wrote a really good post on how WordCamp Ottawa nailed the feedback). Nothing was overtly negative – I was my own […]
Love this. I remember WordCamp Albuquerque did analog feedback forms a couple of years ago as well, and as a speaker I found it immensely helpful to get almost instant feedback. I’d love to implement something like this for WordCamp Denver this year. Glad to see you had a positive experience and what looks like great results!
Thanks Drew! Let me know if you are able to implement this in Denver. I look forward to hearing other camps/conferences experiences in doing this.
[…] Improving Session Feedback at WordCamp […]