As we reflect on the #SQLPASS selection process this year, let’s take a big step back and look at a lot of possibilities. When you’re building a conference or training class, there’s so many ways to build a good agenda. There’s no right way or wrong way – it’s all up to the organizers.
Let’s pretend you’re building an all-new event from scratch.
Will the final decisions be based on numbers or personal curation? Sounds like a loaded question, but there’s absolutely nothing wrong with picking sessions based on feelings. Heck, that’s how we do it as a company – when Jeremiah, Kendra, and I are building a training class agenda, we take some numbers into account, but the agenda is really driven by what we think makes a good flow across the entire event. You want to know this going in, though, so you don’t even bother pretending like the decision is based on math, and it affects some of the decisions you make along the way.
Will you accept speaker submissions from the public? Private-run training events don’t, but most conferences have a public call for abstracts. SQL Intersections doesn’t – Paul and Kim curate the list of speakers, and then let the speakers submit sessions. This is a totally valid “right way” too – I think it leads to a great group of speakers. The key to making that work is that Paul and Kim are heavily involved with the community and know a ton of speakers well. This approach wouldn’t work for a community-driven conference where the organizers have regular day jobs. (Whatever regular means these days.)
What topics will you accept? Who is the training for? I like to write down the perfect attendee’s profile complete with job description, background, and pain points. That profile drives the agenda and the modules. As the event gets larger, like a regional or national conference, the profiles get more and more general.
What are the session requirements? Does it require demos? A minimum percentage of slides? Does it have to have been given before at a smaller event? Does it have to be a world premiere?
Will you communicate these requirements to submitters? As crazy as it sounds, not all conferences do. They want to hear what you’d love to present, and then they’ll decide whether it’s the right fit for their event.
Will you take speaker experience into account? How? Will you take the speaker’s word for it, or will you check references? (Keep in mind that all this takes real work.) If someone has never presented before, will you take them at your event? Would you allow any exceptions, like a Microsoft employee who was involved in developing the actual feature being discussed, or a case study by a real-world DBA?
Will you take past speaker feedback into account? How? If someone HAS presented before, do you want to know if they bombed? How will you get the data? If the feedback was extremely bad due to events outside of their control (overheated room, broken projector), how will you account for that? This is really hard if the other events under your control, but I love how Stuart Ainsworth is taking a first step to help.
Will you take speaker ethnicity, gender, and employer into account? How? If you’re trying to build a more diverse conference, these are tough questions you have to ask.
Will you let attendees vote on the sessions they want? It usually takes weeks for the conference organizers to go through sessions. During that time, you could display the speakers & abstracts to the public and let them vote and comment. Think of it as crowdsourcing your proofreading process. Related questions: who actually gets to vote? Do you display the running votes and leaderboards, or keep those hidden?
How will you take those votes into account? SQLbits lets the public vote, but doesn’t show the vote totals, and doesn’t say that the sessions are chosen by votes. I love this approach because it just uses voting as another component in the selection process. After all, it’s not like any one of the above components purely drives the final choices. The voting may not even influence selection at all – the marketer in me loves the concept of letting people vote because it makes them feel like a part of the process. That probably makes people more likely to attend.
How will you take all of the above into account? Is it a simple percentage basis where each part counts X% towards the score, or do you have showstoppers? For example, if someone has presented before and the topic is relevant, but no one votes for it, is that an instant showstopper?
Who will make the decisions? If the decisions are based purely on data, this is easy, but as you’ve figured out by now, there has to be human intervention because this is seriously hard work with lots of opportunities for exceptions.
How much flexibility do they have to override the data? If a very popular speaker completely bombs the process, do we give him a session anyway?
Will you share the decisionmaking process? Do the speakers and attendees get to know how this all works? (This is why conference selection committees often have to sign NDAs – the conference organizers don’t want to show the public how the sausage is made.)
Will you share the scores and rankings? Do the speakers get to know how close (or far) they were, and what they need to do in order to get closer?
When will the decisions be made? It can actually be the day of the event. At unconferences, the room lineup is a whiteboard where speakers write their session titles on post-it notes. At the beginning of the day, attendees drive the agenda. The agenda can change throughout the day as the discussion encourages new topics. Read more about how unconferences work. I haven’t seen one of these in the SQL Server market yet.
Can people buy their way past the selection process? If someone hands you a check, can they get their own session? Don’t recoil in horror – most SQLSaturdays work this way, and this past 24 Hours of PASS appears to have went the same way, although PASS still hasn’t publicly admitted money was involved yet.
Will paid-for sessions be labeled as such in the agenda? In a rather disturbing trend, purchased sessions aren’t even labeled as marketing sessions anymore. Unsuspecting attendees are walking into these sessions expecting education, and getting a rather nasty surprise.
Will you solicit feedback about your own process? If you don’t share the process, then this question is already answered, and you can expect continuous grumbling about the lack of transparency, hahaha. Deciding to keep the process secret means deciding to accept criticism, and you can’t complain about public complaints. You made your bed.
How will you measure the success of your process? Speaker ratings aren’t just about how the speakers are doing – they’re about how satisfied the attendees are with the session selection process. Given the exact same questions year over year, and excluding venue-related questions (broken AC, broken projector, overcrowded rooms), how did the speakers do? An upward trend here is the single most important thing for a conference organizer.
Building the right agenda is especially important for multi-million-dollar events when their attendees can often only afford to hit one conference in that subject area per year. For example, Ken Wilson writes:
My training budget is thin, so I had to choose one event to attend this year. It was a tough choice since there are so many great events, like the PASS Summit, the SQL Skills Immersion Events, the Brent Ozar Unlimited training, SQL Intersections and many more, but I chose the PASS Summit. Therefore, my experience at the Summit needs to be both an excellent time of community and networking as well as top-shelf training.
Continue reading Ken’s post on the Summit selection process.
You might be in Ken’s exact position – only able to attend one training event per year, or maybe still aspiring to fund your annual learning adventure. How would you want the conference organizers to build the agenda?
There’s no right or wrong answers to all of these questions – there’s just a lot of gray area.