Read this if you’re a #SQLPASS attendee – you’re the one I’m trying to help.
Last week I wrote about how conference organizers pick sessions, and I closed by talking about how there’s no clear right or wrong way. However, some ways are wronger than others, and today let’s talk in general terms about what happens when things go wrong.
I can’t possibly discuss this subject with actual speaker names lest I hurt people. (Disclaimer: that’s a dark music video about drugs and violence, but the song is insanely danceable and catchy, and it’s been stuck in my head for weeks. You’re welcome.)
Enter Speaker 47.
Presenters, if you’re reading this, I want you to really understand that I am making this guy up. You’re so vain, you probably think this post is about you, but trust me, it’s not. However, if you see elements of yourself in Speaker 47 – and hey, I do – that just means your journey to improve your public speaking isn’t over yet, and mine certainly isn’t either.
Speaker 47 is the really friendly, magnetically charismatic person that you like right away. He has a great name that’s easy to pronounce, he doesn’t look intimidating, and he’s always got a wide smile for any stranger who comes bearing a question. He loves technology, and he loves sharing what he knows. He loves helping people, so he knows exactly what kind of title and abstract to write.
You love Speaker 47.
But his presentations and his delivery are horrible.
I don’t mean like kinda bad. Attendees complain that he mumbles through his delivery in a monotone. His demos have a 50% chance of working on a good day, and they don’t convey the material anyway. His material doesn’t match the title or the abstract, and the attendees have a hunch that he’s never actually done any of this stuff in production – he’s just playing with it on his laptop to learn, and they need real help for real world environments.
This isn’t his first time at the rodeo, either. He does this same routine over and over at local user groups and SQLSaturdays. The leaders and volunteers simply hand him his feedback forms. They don’t compile the numbers, and they’d feel too guilty to do it anyway. Sure, they all kinda know he’s bad, but he’s such a nice guy, and he’s working to get better, isn’t he?
And then Speaker 47 gets into the Summit.
Years ago, I wrote about why national speakers should get good feedback locally and regionally first, but that’s not how PASS works. The Summit selection committee has no idea how bad the speaker’s feedback is, so he gets in.
He bombs.
PASS compiles the data, sends out the rankings, and other speakers notice him at the bottom, but nobody wants to say anything – because he’s just so darned nice, and it would be unprofessional to say something. It’s his first year, so you gotta cut the guy a break, right?
So Speaker 47 gets in again.
Even though he bombs, the selection committee doesn’t lock him out of the Summit based on one bad session. After all, people grow and learn, right? Maybe he’ll do better next time.
In an ideal world, they’d look at his session feedback from local user groups and SQLSaturdays after his bad Summit session to see if he learned from his mistakes. In reality, they can’t, because local user groups and SQLSaturdays don’t compile feedback and send it up to the national level. Even if they wanted to, the infrastructure isn’t in place today, but I applaud Stuart Ainsworth’s move to make it happen.
Speaker 47 keeps getting a fresh batch of attendees because for the most part, people choose sessions by topic and title, not by presenter name. Even if you as an attendee wanted to make decisions based on data, you can’t, because PASS doesn’t publish speaker ratings. You’re helplessly stuck in bad sessions, and when you want to leave and get into a better one, the room’s already full – because other people left bad sessions too.
Meanwhile, amazing speakers don’t get in.
They submit, but because the selection committee doesn’t know what great sessions they’ve put on at the local and regional level, they get rejected over again and again. They get burned out because they see Speaker 47 get great sessions – sometimes even paid pre-cons! – and they just give up.
Unless you spend a lot of time watching presentations online and at local user groups, you’ll never get the chance to see these incredible speakers. I’m going to name a name here because it’s such a standout example – I sat in Paul White’s pre-conference last year, and it ranks among the very best sessions I’ve ever seen. This year? Rejected. Part of that is probably his own fault – he only submitted a pre-conference session, and I bet there’s rules about pre-con presenters also having to present regular sessions too. But all of this is locked in the secrecy of the PASS selection process that isn’t explained to volunteer speakers.
And it’s your loss.
This process wastes your time and money as an attendee.
You spent over five million dollars to attend Summit. You deserve not just good, but great sessions, one after the other. You should walk out of there saying, “Holy cow, every session was amazing! This wasn’t like three SQLSaturdays back-to-back – this was like a world-wide, the-very-best-of-the-SQL-community Greatest Hits event!”
But you don’t.
It’s not Speaker 47’s fault. There will always be many Speaker 47s, and we should be encouraging people to speak, not discouraging them. I know some speakers will read this as a warning, and I can’t do anything about that.
It’s not my fault either, although I know a few bloggers who are more than happy to paint me as a whiny celebrity who’s only interested in myself. As I write this, I know full well I’m going to piss off people by demanding a higher level of quality for our five million dollars. For too long, PASS has been about the speakers when it should really be about the attendees.
It’s not the selection committee’s fault. They’re made up exclusively of volunteers who pour their hearts into picking the best sessions they can given the limited amount of resources they have available. We have to get them more resources, and with the sheer amount of profit involved, PASS can afford to do it.
It’s your fault. It’s time for you to stand up for the sessions you want, and to demand quality.
I need you to do three things.
Ask info@sqlpass.org for attendee feedback be considered in the PASS selection process. It’s your registration money – why should you be forced to repeatedly get burned by sitting through Speaker 47’s sessions?
Ask that attendee feedback be made public. You need help making session decisions. It doesn’t have to be the raw numbers – for example, in the agenda, maybe put gold stars next to speakers that got great feedback last year. I’m not saying this to get the gold star either – if this idea goes through, I will personally request that I never get a gold star just so there’s never even an appearance that I’m being greedy here. This is about you, not about me.
Lastly, share this blog post. I’m not in this for the hits – I just want to get this message out to people whose opinions matter most: attendees.
“Unless someone like you cares a whole awful lot,
nothing is going to get better. It’s not.”
Dr. Seuss*
-
- Not a real doctor
68 Comments. Leave new
It’s interesting to me that as data professionals these decisions are made without any data at all. This post has a lot of merit simply because I don’t buy 3 star items from Amazon if a 4 star is available.
Likewise, a teacher rating website was instrumental in my college class decision making process to ensure I was getting the best classes.
Lastly, who cares if you want a gold star, your presentations deserve the recognition (good or bad) and any deviation from the standard would call the whole system into question. Let’s just not hand them out like Oprah.
Ryan – yep, ratings are crucial to buyers. Data is a critical part of life now.
Brent, I’m not clear that that the decisions are made without regard to attendee feedback – I know I’ve heard it being applied to precons for example. I agree that the feedback from SQLSaturday is not being used, and that’s a technical issue as much as anything, as long as it’s on scraps of paper it won’t get used.
I guess I also don’t know how many people fit your speaker profile at the Summit – whats your guess on the pcrcentage?
Appreciate your efforts on this, it’s a good conversation to have.
Andy – I’m in a tough spot to start naming percentages. I figured I’d start by posing just a single example to say, “Is this OK?” If I start naming names, things are going to go downhill. (Heck, I already know I’m going to get thrown under the bus just for writing this.) I’ll steer clear of anything bigger than that for now. 😉
Maybe I’m remembering wrong, but I seem to recall that they clearly call out that pre-con presenters have to be available to do at least 1 regular session too. I don’t think you can call it a secret rule when they explicitly call it out.
Robert – thanks. Can you point out where that is?
And I’m kinda remembering that same language – being available to do a regular session is different than submitting one.
I can’t because the call for speakers isn’t currently open. But if you don’t submit regular sessions, then how can one of your regular sessions be selected as well.
For most European conferences I’ve done this year, we’ve agreed on pre-cons first, and then they’ve selected sessions afterwards. After all, folks who travel internationally sometimes can’t afford to fly over to give a single session for free. (Remember, PASS pays no travel or hotel expenses, unlike every other major conference.) Some international presenters can only afford to present if they get a pre-con, so it’s not unusual for other conferences to arrange that first. (SQLbits, both recent Rallys, SQLSaturday Portugal, etc.)
Can’t recall if it was explicitly stated this year, but I know I’ve seen that before in either the speaker terms (the ones you have to accept before submitting sessions) or somewhere else in the submission process. It’s definitely been stated explicitly before.
I couldn’t reply to Brent’s reply, so I’m replying here.
I agree with the process of selecting pre-cons first and then doing the other submissions later. It helps nobody if a speaker can’t attend unless they get a pre-con, but they get just a regular session that now has to be cancelled.
Pre-con requirements were actually quite strict circa 2012 (cant recall about last year). Far more so than you (Robert) mention. Strangely, this year they appeared to be lifted, however the selection process for those really is obfuscated. This probably means that any pre-con submission considered outside the scope was just binned. There should perhaps be more transparency for precons which is even outside of the Programme Committee as I understand it, and I would be in favour of re-introducing a flexible set of “requirements”. For instance if someone is quite clearly of Paul White calibre, it wouldn’t matter for a second whether they fitted into a certain set of requirements. Awesome is what awesome does.
Mark – yeah, pre-con requirements used to be more explicit (like what kinds of events you needed to present prior to stepping onto the world stage). It does look like pre-con selection became more subjective this year, which seems odd given the constant “we are the community” drumbeat coming from PASS, but … I’m okay with that. If the PASS leadership wants to change the selection into a private, personally curated affair, that can totally work great too. I think it works great at SQL Intersection when Paul and Kim curate sessions, for example.
I just think that if we’re building a community event here, then we need to listen to the community.
I appreciate the tough spot, I’ll infer you think it’s significant:-) I’ve looked at the attendee evals over the years and there are some Top 10-20 scorers (you, Adam, Paul), then a lot of people in the middle, and a few that really fall off on the bottom.
Andy – HA! Thanks, sir.
Great read, well written. Thankyou
Heidi – thanks!
I am (at least in part) Speaker 47 given the fact that I take what I’ve done in production and try to push it to a logical conclusion (on a laptop, no less – the HORROR). I think that the point you made about Speaker 47 needing encouragement (and mentoring?) at the more local level is spot on. Perhaps a more intentional process of mentoring Speaker 47s should be initiated at the local UG level? A local board member in charge of speaker development?
I’ve been in Speaker 47’s presentations, and I want those hours of my life back. After attending several big events, there are speakers I avoid even if they’re the only session, and there are speakers that I don’t care if they’re reading the phone book, I’m going because it will be interesting. As a speaker, if you had 50 attendees and got 5 ratings, the silence is deafening about your performance.
Speaker ratings are one thing I loved about speaking at VS Live. We got those very quickly and they were helpful to me. You always have to balance the euphoria of the “awesome presentation” with the attendees who didn’t have the same experience.
SpeakerRate and a couple other sites have tried to centralize a speaker’s performance, I I used to encourage my audience to go there with very little success. But the idea of a central ratings entity, to me, has a lot of merit. Maybe the answer is for PASS to team up and make a service like this part of both the speaking and attending experience.
Rich – that’s a great point about ratios. Plus it can indicate that people started walking out after the presentation started bombing or it didn’t match the abstract.
Also, attendees have to be honest in their ratings. I’m a terrible offender – “he was brave enough to present, I’ll give a good evaluation even though I wasn’t impressed”… But that doesn’t help either that speaker or a selection committee, so I need to be more thoughtful with my evaluations.
Lisa – yep, agreed, and us speakers want the honesty too. We really want to get better, and no one else will be honest with us when we don’t do well.
Brent – you make some good points, and I add this at the risk of trying to ride my pet peeve on the coat tails of yours, but is this not a good case for open sourcing the SQL Saturday website so some dev could implement this feature? I made a general case for doing that here: http://www.justaprogrammer.net/2014/04/09/the-case-for-open-sourcing-the-sql-saturday-website/
So, lets just say one of us wrote the code to make SQL Saturday feedback get collected, that would lower that technical barrier of entry of implementing what you want. If we were voting on a github (or codeplex or whatever) issue with a patch, then that’s one less reason not to implement it. Someone wrote the damn code, we just gotta deploy it is a powerful argument.
Justin – I’m actually a believer in not reinventing the wheel. This is just me, but I’d rather see the agenda and feedback software outsourced. This isn’t a core competency for PASS. We wouldn’t write our own accounting software, so why write this? Of course, if we do need it to be a core competency, then sure, open source is the way to go, but keep in mind that it’s not as simple as letting anyone commit. We’d need a commitment to QC too.
Software like this exists already–look at Eventboard from Falafel Software.
Rich – agreed, lots of software exists. That doesn’t mean it’s free or the right fit. 😉 Someone still has to do that kind of digging.
Justin, I’m not in favor of open sourcing the framework. In part because it’s IP that PASS owns, in part because the framework is less the problem than someone having a shared vision of where we need to go. It’s not a simple conversation, but it’s not as simple as saying just change it either. The issues Brent talks about go far deeper than anything in or not in SQLSaturday, and I’m not yet convinced that evals there would change the issues we’re discussing.
Only problem I see with ratings is the “celebrity phenomenon” which is where something based exclusively on that will be heading. If you claim people is nice enough that won’t be overly critical even when bad is happening, there is also some level of fanboyism that will happen when the flamboyant and more show oriented presenters will be rated higher than those that rely less on those kind of devices. Which probably means you need to put these 2 systems together but not rely entirely on one or the other. Just my 2 cents
Daniel – PASS already has a fix for that – each presenter is capped at two sessions. Even the most flamboyant showman will still only get 2 max.
And what’s wrong with the celebrity phenomenon? If a lot of attendees want to see someone present, that means they’re doing a good job. People don’t clamor to see a bad session.
Obviously I’m biased here. 😉
🙂 You are bringing the topic to good or bad. That is not the celebrity phenomenon. That is when something (good or bad) gets popular just because of a mob mentality and the suppression of critical feedback to give in to sentiment based reasoning, the very same thing you used to support your argument about speaker 47. Plus you are so vain if you thing the flamboyant comment was about you :D. Great post BTW and lot of b… courage to say things like this which are not easy to discuss,
Personally I agree, that throwing certain speakers into the same mix for selection is probably not the best system for the biggest SQL conference in the world. The following should get picked at any event period (to name a few): Kevin Kline, Aaron Bertrand, Grant Frichey, Paul Randal, Kimberly Tripp, Paul White, Adam Machanic, Andy Leonard, Marco Russo, Kalen Delaney (and I could go on). I think we would obviously get into a situation whereby no one new could get a session so the “select” list would have to be capped and have high selection criteria. Where you set that split at is open to debate, but a premium conference does deserve premium speakers. Removing many of the premium names from the Programme Committee would also hugely offload the work required and provide a more rigorous examination of each session and speaker, however I think you would be surprised how much time and effort this process is. And as a final point, unless I am speaker 47, I have only ever seen one full speaker who I thought was atrocious and should never have got a slot and can only think of a few who I would question their continued selection -and even that is being a little harsh. As I’ve said over twitter I think the content at PASS is by far some of the best quality material out there, with very few exceptions.
One point we absolutely agree on I think is that SQLSaturdays need to start taking this feedback and pass it up to HQ. Hopefully Tim can sort something out in this area going forward.
Mark – as much as I love all of the presenters you name, I think it’s dangerous to give guaranteed speaking slots. After all, sessions are a combination of speaker AND material, and it’s possible that some material just might not call to enough attendees.
I’ll give a real life example I saw recently – an absolutely amazing Microsoft speaker had a session at a smaller conference. His session was on Azure, and he had 2 attendees – both of which were MVPs who just wanted to hear him talk about anything. As good as he is – phenomenal, really – sometimes the topic just doesn’t work for the attendees.
About you not seeing bad material – you just named a great list of speakers, and you’re a little more in touch with the speakers than the average attendee. Is it fair to say you might be a little better at picking out good sessions and avoiding iffy ones?
🙂 you are probably partially right about me only seeing great speakers because I tend to gravitate to their sessions. I do however try to listen to many of the recordings and frequently discover many gems who afterwards feel embarrassed I didnt know of, like the excellent Meir Dudai. I’m also on the lookout for new speakers for my event and last time I selected someone after first saying to the team “who the hell is this guy?”. I managed to find that he had spoken at one single SQLSaturday and took a punt on him. He turned out to be not just one of the best speakers I’ve heard, but one of the nicest guys too. Offered him a guaranteed precon in the future based on this. Imagine a system though, where I could have simply consulted a central database and pulled up his session ratings. Only danger is if he had had a bad day at the office during his first presentation, there is a danger that emerging speakers would struggle to be born. Obviously your point is (and I agree) that for the Summit, these speakers need to be ready and experienced …and consistently good. I actually think that PASS could do much more with lightning talks for first timers, that would hugely help in preparing some of them for the big stage.
Mark – yep, agreed. I think the SQL Server community is doing a great job of encouraging others to get onstage (I certainly push hard for it) and we can keep doing better. There’s so many great voices out there.
Brent – I get where your coming from on this but Very few SQLSaturdays actually collect the feedback and even fewer user groups collect any data at all about there meetings.
In terms of speaker feedback. I would definitely love to see attendees providing more than just a rating. Most attendees don’t leave any comment as to why they have a particular rating. This info would be invaluable to a speaker or conference organizer
Brandon, you’re right, and the truth is that it’s a cultural thing in chapters & SQLSaturdays that has to start somewhere. It’s tough, because if you’re a smaller chapter with a smaller SQLSaturday, you don’t want to critique people who are putting their necks on the line to talk; at larger chapters and events, most chapter leaders are swamped with other operations, which makes it tough to collect data.
A first step is always awareness, but a second step needs leadership at the local level to make it a priority. While I think PASS can help (speaker’s bureau? standard evaluation recommendations?), it has to be “bought in” at the local level.
Stuart – agreed, but at some level, there may also need to be minimum requirements to run a SQLsaturday, and gathering feedback might be one of them. Given the criticality of Summit quality control, the answer might be for PASS to set aside a certain amount of money for a paid volunteer (so to speak) at each event to make sure feedback forms get turned on and digitized, or that the speaker feedback forms are exclusively digital. It’s definitely going to take more resources either way – we can’t force the already overworked local organizers to tackle a task that’s really critical to PASS.
If ratings from SQLSaturdays were a requirement for speaking at the Summit, I bet the SQLSaturdays that don’t collect ratings would lose speakers thus compelling them to start collecting ratings.
Robert – that’s actually a good point – many organizers also want to speak at Summit themselves, so that might create the incentive right there.
Robert, you are right. This would be a game changer for SQLSaturdays especially who have to compete against other national (non PASS) events like me. It was/is something I was going to raise at the Summit for the SQLSaturday Leaders meeting. Quite honestly it should be very easy to implement imho.
This is what I want at the Summit each year –
https://www.youtube.com/watch?v=bMbtzalS3u8&feature=kp
Simply the Best…
I’m so glad I took that shot of her at the top of the post, capturing her mid-stride. I laughed pretty hard that day.
(But please note – that doesn’t really mean I want to just stay in the arms of Paul White, Paul Randal, Brent Ozar or Erin Stellato… Well maybe Mr. Randal…)
Thought-provoking post, Mr. Ozar. Although I don’t like that you used me as an example. 😉
Allowing continued feedback from SQL Saturdays and UGs – as you propose – should mitigate a scenario where Speaker 47 is locked out after a single bad delivery. And it allows the SQL Server Community a voice in the process, which is what I believe some seek when they request session selection via voting.
I like it.
:{>
Andy – thanks sir! And I feel bad about using you, but I did at least leave the pictures out. 😉
Andy,
I remember attending a session on running a user group at the first summit I attended. You’re Speaker #1 in my books.
D’aaaaaaaaaaaaaaaw.
I agree with you 100% Brent… the speakers should be scored by “pounds of bacon” based upon prior reviews so that we are ensured to have the best of the best at PASS. It’s frequently disappointing to enter a PASS session with unknown speaker and find you’re in “Speaker 47″‘s panel. Yuk. Especially when I am the one dropping several thousand bucks to attend (as opposed to attending on the company’s tab).
LadyRuna – that’s a great point – between the travel, hotel, and vacation costs, an attendee is easily paying hundreds of dollars per session.
Brent, thanks for writing this though-provoking post. It has resulted in good follow up discussion, with lots of good points being made.
I like any initiative that will increase the amount of feedback I receive (and attendees, please: use those free-text fields, they are for the speakers the most relevant part of the feedback!).
I also think that using previous feedback to increase the quality of high-end events is a good idea – but beware that you don’t run into a situation where a speaker will never make the next step. The Grand Slam tennis tournaments use wild cards, and I think PASS should too.
However, there is also a side to the discussion that I miss in these comments. There is a difference between purely commercial events such as Connections and Intersections, and community events such as the PASS Summit. That difference has many effects – but the one that affects me as a speaker the most is money. Except for the precon speakers, presenters at PASS also make a considerable investment. (Which is the reason why I post this comment here, attached to Lady Runa’s comment about cost).
I consider myself lucky and honored every time I am selected to present at PASS (third time this year), but I have to make a deep financial investment. Yes, I do save a lot because I get a free registration, but other than that – I have to pay my flight (from Europe: prices vary but it’s usually over $1000) and hotel; I have to take time off from work (I am lucky to have an employer that donates half the time from the training budget, but it’s still 2.5 days less holiday time for the family). And because I am from Europe and 95% of attendees are from the USA, I could never justify this as “driving future business” when I was a freelance consultant (and I can use that justification even less now that I am on payroll).
This is just me. Other speakers have other stories. Some may come from Portland and get contracts from speaking. Others come from even further than me (South Africa, for instance) and hence pay even more. But for most speakers, speaking at PASS is a cost factor, done for the love of community, not a job or a PR activity.
The arguments given about how much money attendees pay and that they should expect value for their money – they are all true. And for events like Intersection and Connections, that’s all there is. But PASS has the unthankful task of having to make sure that attendees get their money’s worth of content – but also making sure that speakers are rewarded and recognised for their commitment and investment.
Hugo – thanks for the comment.
To make sure I understand – you wrote that PASS needs to make “sure that speakers are rewarded and recognisd for their commitment and investment.” What specifically are you asking PASS to do?
Are you suggesting that PASS pay speaker travel and hotels the same way that other conferences do? (I would understand that request, but I’m not sure if you’re asking for that or something else.)
Thanks sir!
You’ve got me there, Brent. What am I asking PASS to do? Honestly, I haven’t the foggiest idea.
My comment was a reaction to the tone of your post and some comments that, in my opinion, were focusing TOO much on the attendee side. The argument is used that attendees pay a lot of money to attend. This draws a one-sided picture if one does not also draw attention to the significant investments made by speakers. (Less than attendees if you look only at out of pocket cost; perhaps even more than attendees if you put a dollar value on the hours spent preparing sessions)
So what am I requesting? Not much, just that people in the discussion realize that, unlike some other events, speakers have to pay money to go to PASS.
(Oh, and yeah – I’d love to see PASS paying speakers compensation for travel and stay, But that is a rather selfish request: I’d love to never miss a PASS again but simply cannot justify the money, and this would give me an out. But I’d probably then start to whine about whatever part of the PASS budget is cut. Let’s not go there, this is a separate discussion)
Hugo – I don’t think I quite understand the connection between the speaker reimbursement and he speaker selection process.
Attendees are paying over five million dollars for a good product.
PASS chooses not to reimburse speakers, and keep more of that profit to fund things like new analytics conferences, regional SQLSaturdays, and PASS HQ staff. The real person funding that effort is you, the speaker. If you want to be recognized for traveling to Seattle in your own expense, you shouldn’t take that up with the attendees. You should take that up with the company that chooses to use your travel and hotel money to fund their organization.
Attendees have no control over that. That’s a decision you made, to agree to let PASS take that money from you. I’m not saying that’s wrong – I let them take money from me too, and I know what my work is being used for, but I can’t tell attendees, “You should accept a lower quality session from me because PASS needs to give money to PASS HQ.” Those two things just aren’t related.
Also, just to be clear, you said the speakers make a significant investment. I strongly disagree.
In a room with 100 attendees that each paid at least $1,500 to attend plus say $2,000 for travel and hotel, and a single speaker that only spent $2,000 for the travel and hotel, the total expenditure is $352,000. The speaker spent $2,000 of that. That may be significant to the speaker, but that is less than every single attendee in the room paid, AND less than 1% of the money spent in the room.
You’re a special case because you’re international, but again – so are other attendees. All of us spent our own money to be there. You got in for free, so you spent less than everyone else in the room. No one is going to feel sorry for you on that one.
Thanks for the response, Brent. You are forcing me to think about this, and now the foggy ideas in my head are starting to form a clearer picture.
First: I disagree with your arithmetic. Every person is equally important. If I mess up a session with 200 people in the room, I feel just as bad as when I mess up in front of just a handful. It doesn’t add up.
But even without the multiplication, we have no disagreement that the value for attendees should be top notch. Admission for the summit, even with early bird, costs a hell of a lot of money. Attendees can and should expect to get value. Organizers and speakers should constantly be aware of that.
But there IS relevance to the fact that PASS does not compensate speakers like Connections and Intersections do. PASS choses to use the Summit to fund other activities. To maximize the effect, they have cut out one of the significant cost factors for other conferences. The only reason they can do that, is that there are sufficient speakers who, even without compensation, still want to speak. And that those speakers are of sufficient quality. As long as PASS holds on to this business model for the summit, this pool of speakers is a major asset that they should treasure.
Every PASS director reading this post and this discussion should immediately ask themselves: “If we were to set up such a system, would it have any detrimental side effects? Is there any chance that people who are considering to become a speaker will not do so because they are more hesitant to go out there knowing that if they mess up, the bad reviews will go online? And is there any chance that speakers that are JUST good enough to take the next step and present at the Summit would not submit because they think their ratings won’t meet the bar?”
Growing community is an important task of PASS. Encouraging new speakers is a major part of that task. I am afraid that the suggestion to publish speaker ratings and use them for session selections may backfire, causing budding speakers to rethink, or growing speakers to be more hesitant to take the next step.
That being said, I do also believe that the content of the PASS Summit has to be top notch, and that speaker quality should be taken into account when selecting sessions (not as strict as purely commercial venues, since speaking at PASS can also give a “good enough” speaker the push to step into “great” territory, which helps toward the long-term goal of growing community).
Your idea has potential. It also has risk. I do not think it should be rejected off hand because of the risk. But I also think it should not be implemented without further thoughts because of the potential. Risk and potential should be weighted, maybe resulting in elements of your idea being amended and implemented and other elements being rejected.
Yes, that is vague. I do not know the answers, I do not have the ideal solution. Just sharing my thoughts.
(Maybe we should discuss further over a beer at the Summit? First round is on me!)
Great response! You’re on, and the next round is on me. 😉 I look forward to seeing you again at the Summit, sir.
Oh, and I would also like to point out that I am not looking for people to “feel sorry” for me. It’s my decision to spend the money to go to PASS (and many other conferences) as a speaker, and I stand by that decision without asking for pity.
Every speaker has their own reasons for speaking. Some try to grow their business, some try to get to the level where they can ask money to present. My reasons are different. I speak for two reasons, one altruistic, and one selfish.
The altruistic reason is that I want to help people get better, that I want to share my knowledge, and that I want to give to the SQL community.
The selfish reason is that I simply love standing in the spotlight and getting that applause at the end of the session. 😉
Great post with a lots of good points specially about the wasted time and money but – maybe because it hurted my Speaker 47 feelings – there will always be sessions that get higher ratings than others. If every session get max ratings what’s the point in having the ratings. Just like when prioritizing project requirements and every requirement is top priority.
Let’s get some mentoring started maybe by looking at those ratings and having the top 50% coaching the other half…
I definitely would like to work a model out for SQL Saturdays where there is a centralized and consistent way of doing speaker ratings. Any suggestions on this are welcome.
Regis – hi, good to meet you virtually. I’ve done a *lot* of writing and speaking on how to improve presentations. When you suggest coaching, I definitely agree – that’s what I’ve been doing for *years*, and I’ve seen a lot of efforts by other speakers as well. If you’re looking for resources I’ve written on how to present, check out this list for starters:
https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=presenting%20site%3Abrentozar.com
Enjoy the journey!
Brent
This is the reason I was ready to skip PASS this year. Last year I got to enjoy 3 sessions tops and the rest were just a big disappointment. I don’t think you are being whiny and appreciate that you are giving examples of what can be done better instead of just pointing out a problem. Too bad we can’t take the top rated speakers from each SQL Saturday and have them submitted to PASS instead of it just being an open forum. In high school, you couldn’t present at nationals unless you were the best at your local level. Anyone was welcome to watch the presentations, and there were trophies!
Love the song btw, thanks for getting it stuck in my head. Here is one back http://youtu.be/7PCkvCPvDXk You’re Welcome!
Andrea – dammit, that is really addictive. I’m going to be humming that one tomorrow, hahaha.
Dang, I’m sorry to hear that about the sessions. The high school analogy is a good one – to some extent, speaking is kind of a sport, too!
[…] Warren (@sqlandy) wrote a blog post about a speaker challenge and Brent Ozar (@BrentO) wrote about Speaker 47. Erin Stellato (@erinstellato) responded to An Open Letter To SQLSaturday & User Group […]
PASS has an online Session Evaluation for the Summit sessions. I’d love to see it expanded to include SQL Saturdays and User Group Meetings as this would consolidate all the relevant speaker review data. Then the Program Committee would just need access to that data.
Perhaps make the chapter level optional for new speakers that are just starting out. That way if they totally screw up their first few presentations, it’s not factored into the equation.
New speakers need to be given a chance to learn how to speak and improve. But if they can’t do the latter, the data should reflect that accordingly (which would be happen when they start getting feedback for their SQL Saturday presentations).
I don’t see any perfect solutions on the horizon, but if you’re always looking to improve the process/system, you’re on the right track.
Matt – ooo, I like that idea of the first X presentations don’t count for scoring. Of course, at the same time, we’re then also saying you can’t get into PASS without having at least Y scored presentations plus X unscored ones – possibly raising the number of required sessions rather high.
(I don’t think that’s necessarily a bad thing – the Summit really isn’t where new speakers should get their first shot at public speaking, not when attendees are paying thousands of dollars.)
To have the benefit of not being “forever” weighed down by a failed first attempt, without having the problem of requiring X + Y prior presentations, you could use weighted averages, with the more recent feedback scores weighted higher and older sessions weighted lower (and eventually disappear completely).
This also has the benefit for attendees that a speaker that is over his top and start to get consistent lower grades will be faster recognised as such.
My first PASS Summit I didn’t know who the big names were. I was chatting at lunch with someone about a dreadful session – how the presenter spoke in a monotone, was difficult to follow, didn’t make effective points, etc. The person asked who the speaker was and when I named him, jumped back saying “oh no, he’s really smart, he knows what he’s talking about, blogs all the time…. BUT none of that negated the fact that he was a terrible presenter. I expect that he got better feedback than he should have, and kept on getting sessions, because of who he was rather than how effective his sessions were.
Another year I attended a Summit session by a big name presented in one of the large halls in front of hundreds. It was the least professional session I’ve ever seen, with the speaker acting like he was such a star that he actually could read the phone book and we’d all be impressed. He casually talked about changing out slides moments prior to the session, rethinking what he’d talk about, and doing questions and answers with attendees that he knew regarding how great his paid services were. It was embarrassing. I flamed through my feedback form, furious about it. Most of the content eventually presented was valuable, but the attitude killed me.
Big names in the community should not get spots just because of who they are if they don’t also have good presenting creds, and big time presenters should be reminded to always do their level best to bring their A game. Serious money is being spent and speakers who don’t respect PASS enough to give their best possible talk for free shouldn’t submit sessions.
I’m not sure of the best way to eliminate either of those kinds of bad sessions, but I agree wholeheartedly with Brent and those commenting about SQLSaturday feedback re: up and coming speakers. It really really really should be electronic, and used by the Summit selection committee.
I attended the first ever SQLSaturday in Maine in June. It was a good event and I was so thankful to the great presenters who came all the way to Maine to speak. There were some glitches, of course, but the feedback forms were the one event failing that bothered me. They were little paper strips that most people didn’t seem to remember were added to their bag. Clean, quick, universal electronic feedback needs to be implemented asap.
Anne – I’m sorry to hear that, but I’m not surprised. I had a similar experience at my first Summit – I sat in a session by a presenter whose blog I’d read for a couple of years. I knew the presenter knew his stuff well, but the delivery was pretty rough. I remember overhearing a couple of nearby attendees complaining, and I kept thinking in my head, “Don’t you know who this guy is?!?” But you’re right – names don’t matter nearly as much as delivery, and I was probably one of the few happy attendees that day.
The A game thing matters on a lot of levels. For example, I know I miss out on a lot of PASS parties because I won’t stay up late the evening before I present. If you’re hung over, you’re not going to do a good job presenting. It’s not a matter of chest-thumping or “manning up” – it’s a matter of bringing your best work to people who paid thousands of dollars to be in the audience.
[…] that structure is so fundamental to a good talk; and to be honest, I would feel a bit like a speaker 47 if I […]
[…] at winging it, because I believe structure is so fundamental to a great talk. I would feel like Speaker 47 if I […]