Speaking at a national event is a privilege, not a right. Attendees spent a large amount of money to come to a big event. The event organizers need to ensure that every presentation and every presenter are worth the attendees’ time and money. But how can we gauge quality when hundreds or thousands of abstracts get submitted every year? Simple – gradually:
Before speaking at a regional event, you must have local experience first. In terms of PASS, this means that I would require that speakers do presentations at local chapters before they can be accepted at a SQLSaturday. I’m not saying the exact same presentation should be given at both events, but they at least need to present in both places. Speaking in front of 100 people is very different than speaking in front of 10 people – you have to gauge audience interest differently, control hecklers differently, and project your voice & personality differently. The techniques I use with a handful of people don’t scale to a packed room.
This point is especially near and dear to my heart because I keep getting desperate pleas from user group leaders who need local speakers. I get the feeling that some speakers are just parachuting into big events rather than putting in their time polishing and preparing their work at the local level first. Local chapter leaders need our help.
Before speaking at a national event, you must have regional experience first. This dovetails perfectly with PASS’s recent acquisition of SQLSaturdays, and it does two things. First, it ensures that speakers can perform in front of big crowds, and second, it helps SQLSaturdays maintain a good speaker list.
Ideally, if you get lots of negative feedback at a smaller event, your submissions aren’t favored at national events. This takes a lot of work, but in a perfect world, I’d have a single feedback system (maybe SpeakerRate.com) that centralizes ratings. If you can’t average, say, 4.0 (on a 5-point scale), you don’t get into the Summit.
This topic is very important to me because I remember being very let down at my first PASS Summit. I remember sitting through some presentations and saying, “Jeez, how the heck did this person get approved to speak? Sure, the material is interesting, but damn, they can’t speak. They have no stage presence, they’ve clearly never rehearsed this material, and they don’t know how to answer audience questions.”
Running a national event means making sure your attendees get the most bang for the buck. Vetting speakers like this will help ensure that attendees keep coming back for more.
39 Comments. Leave new
Agreed. At Microsoft’s internal event that I just spoke at there are “track” owners who compete for the best scores all up from their speakers. Speakers have to do tech checks, and they check your previous scores. Because of this competition at the event-staff level, we have really top-notch presentations. The only issue is being able to attend them all!
Great Topic Brent, As a UG leader and working with National Conferences in the past I can not tell you how much I agree with you.
Speaking at other small events also helps with networking in a local area and building your resume at the same time.
Average 4.0? I usually average 3.8 and the overall average is 3.6 with a median around 3.7 at the (count them) three large conferences I’ve spoken at. 4.0 is pretty tough.
Other than that, I agree with everything you said.
Grant – doh! Sorry, I was thinking on a 5-point scale because we’re using a 5-point scale to give instructor feedback at the MCM. If it’s a 4-point scale, yeah, I’d say average 3.4-3.5 or whatever. If you’re lower than that, no dice.
Oh, OK. That makes sense then. I think you & Paul Randall were the only ones that average 4.0 at the last Summit. Seemed like a slightly high bar.
Yeah, I’m not trying to set a high bar by any means. I just want to give speakers a reason to pay attention to their feedback scores, and stop the really atrocious ones from getting a national level spot for the wrong reasons. I’m trying to give PASS more metrics and more valid reasons to say no quickly. Think of it as a very coarse filter that requires little effort. I want it low enough that it only filters out, say, the bottom 25% of speakers according to local and regional ratings.
Brent,
It’s like Baseball. You need to work your way up through the minor leagues first before you get to play in the BIG LEAGUE.
I agree. I’m not an experienced speaker, and I don’t think I’m ready to speak at the PASS Summit where people are paying money to see the presentations. At least not yet. I’ll stick to presenting at SQLSaturday for the time being.
I think it’s a great progression, but I’m not sure it’s realistic to do this year, but sure in the future I think it’s a win for everyone.
I’d like to see a similar progression for becoming part of the leadership team. Run a chapter, be on the leadership team for a regional event (SQLSaturday), and then run for the board. You could have some leeway as far as experience on another non-profit board like Salvation Army or something.
While it would be nice to present at local user groups first, it’s not always feasible if there isn’t one in your area. Driving two hours away is far easier on a Saturday than on a weeknight and presenting virtually is a whole different animal. I do agree, though, that you should be experienced at as a presenter prior to presenting at a paid event.
Just my two cents…
Lori – so, here’s my tough love: if there’s not a user group in your area, you need to travel. You’re going to travel to speak at the national event too. If there’s not even a local user group in your area, that makes me even more convinced that you haven’t had a local mentor to help improve your speaking, and you haven’t had the chance to see a lot of good speakers in-person. The odds are even more stacked against first-time success in a room full of strangers.
I think you have to take into account other avenues for speaking as well. I’ve only presented on SQL Server a few times, but I’ve spoken in front of groups (up to about 200 people) at least 75 times.
Ideally yes, you’ve done the SQL Server path, but I don’t know that you’d exclude someone with other speaking experience.
I have mixed opinions about that. I think it’d be okay for the Professional Development track, but not for the rest. You’re right in that it’s probably good enough, but this is me being selfish – I would enforce the local-regional-national progression because it’s good for PASS. Local groups need speakers, and this is one way to help with that pain.
I’m going to have to agree with Brent here. It’s totally different speaking at Toastmasters vs. presenting to a user group vs. preaching a sermon. Yes, there are key skills that reach across all three, but just because you are good in one area doesn’t make you good in the others.
I’m more in agreement with Jack on this one. I have lead training classes in various technologies for 10-20 people for decades (and that’s hard to say). But I think the time I’ve spent as a Cub Master in Scouting was better preparation for presenting on the international level than all those other teaching assignments.
Thanks for the advice and tough love, Brent, and I completely agree for folks that have never had the chance to speak in public before. Having to travel might keep people that would be great from even starting, so maybe looking at local opportunities like Toastmasters, churches, volunteer organizations would provide similar experience.
Fortunately, like Jack, I’ve had plenty of public speaking opportunities in addition to teaching Computer Science. We’ll see if I crash and burn in Phoenix 😉
You’ll do GREAT! I believe in ya. That’s the only tough love I’ll ever give ya. 😉
And here’s where Brent and I disagree with respect to SQL Saturday. If I remember the conversations I’ve had with Andy Warren, one of the reasons SQL Saturdays were done was to give folks their first chance to speak. It is about free training and growth of the community. Part of that is getting new speakers up and going.
We ran our SQL Saturday in Boston on exactly that rule. If you want to speak, you’re in. We only limited the number of sessions people could do, but we accepted everyone who applied, uncritically.
That said, we a couple of really horrible speakers. But I’m sure they learned from the experience same as anyone.
But what’s your goal? Is your goal to help speakers get experience, or to have a kick-ass SQLSaturday? If your goal is to help speakers get experience, there’s better ways to do it.
This is just me, but I’d rather have 5 killer tracks than have 5 killer tracks, 3 decent tracks, and 2 crappy tracks. I don’t want attendees walking away saying, “Man, that SQLSaturday had some good stuff, but it had some really crappy stuff too.”
I’m even more convinced of it sitting in the MCM training. There are some speakers that make me say, “Dude, whatever you want to present about, keep right on going, because you’re amazing.” There’s other presenters that actually make the slide decks WORSE – I’d rather just read their decks in my spare time than spend my valuable daytime listening to ’em.
The goals of SQL Saturday (http://www.sqlsaturday.com/about.aspx):
Our goal is to provide the tools and knowledge that groups and event leaders need to organize and host a free day of training for SQL Server professonals. At the local event level, we have a broader mission statement:
•Build the membership list of the local user group
•Provide local SQL Server professionals as well as those new to SQL Server with an event that combines training and networking
•Find, grow, and feature local speakers – the next generation of talent.
That last one: find, grow, and feature local speakers means you’re giving opportunities to first time speakers.
I agree with Brian that I think the spirit of the third goal is new speakers, but I think you can interpret it to mean provide user group speakers a larger platform to speak at as well, so Brent’s progression idea could fit that. DO a user group then a SQLSaturday fits into the Grow and Feature part. The Find still happens because locals know that they need to step up at the user groups first.
Yeah, but those goals date from when SQLSaturday more or less competed with PASS. IMHO those goals should change with the PASS acquisition. It doesn’t make sense to have local, regional, and network groups that aren’t aligned with common goals. For example, the national event shouldn’t have a goal of promoting and growing local speakers either.
I don’t want to see SQLSaturday turn into a mini-Summit where speakers are picked solely on name/experience. We have to have a way for new(er) speakers to get experience. I think the piece we’re missing is the paid regional event, something that fits between SQLSaturday and the Summit. Event at a SAT it can be a balancing act, but I love being able to include as many speakers and take some risk on quality.
I agree with making sure speakers “progress” to a big event like PASS Summit. But “progress” is the key – “local” and “regional” speakers need audience and expert feedback to get better.
I’ve really just started – I’ve only done ten or so mostly to user groups. I’ve gotten almost zero feedback from audiences at UGs compared to the one conference I did. That makes it very hard to know if I’m presenting well. I’d like to see PASS’ chapter websites designed to help facilitate managing their meetings – including an easy way for admins and users to see past and future talks, speaker info, host the related downloads, and have a feedback mechanism with open results. I’m going to try to speak at SQLSaturday in Portland – that’ll be a big measuring stick for me… assuming I get some feedback.
What I’d like to see for any speaker “new” to the Summit (regardless of criteria) is some kind of a mentoring program. Like someone said over on Allen Kinsel’s blog, you’d like the speaker to have presented the topic before – but the lead time is really too big to require that. What you could try to do is to help the speaker get a local or regional venue to present the topic prior to the Summit, and make sure some “experts” evaluate the session and help the speaker smooth out the rough spots.
Video tape yourself. Then sit down with peers you trust to tell you the truth and review the tape. Even if you don’t get direct feedback from the user group, you still will get constructive advice as well as the appropriate atta-boys from the folks within your private circle.
Next user group, think about what you need to work on. Pick one specifically. Again, video tape. Repeat the process.
Brian – I like that process. I will give something like that a shot.
Camtasia/Camstudio is also an excellent way to do it, you can start/stop, look at various parts of the presentation without getting over focused on the body language (the least important part in my opinion).
My fear is not getting honest feedback. Like Brent, I want to see good speakers at PASS. I’ve sat through some pretty tough sessions on great content but deliveries that stunk.
So with that in the back of my mind, I set out this year and spoke for the first time (Outside of a corporate setting of 10/20 or a church setting a few times to 150 or so). I spoke at two SQL Saturdays (Boston – hope I wasn’t one of the horrible ones, Grant! and Charlotte).
Boston was my first, still feel I did alright. I got good feedback from people in the halls and a couple in e-mail. I wasn’t on speaker rate then and I received no negative feedback in any form. I know where I screwed up and worked hard on that for Charlotte.
Charlotte came around and I gave the same two presentations, signed up for speakerrate, etc. At my speakerrate site I received only two pieces of feedback (from the same person) http://speakerrate.com/mikewalsh it was positive feedback. I also received some e-mails from folks who attended and one gave surprisingly positive feedback (said I presented like someone else who I really admire and think I don’t present nearly as well). I got no negative feedback but I didn’t receive a lot of samples….
I wonder.. Do people really leave negative feedback? I wish they would, it would help me know where I should focus my improvement.
So.. With the feedback I got at Charlotte, the confidence I have and the laughs I got from the slide as well as some good follow up questions I have convinced myself I am ready to try one of the talks at PASS. Submitting it (I submitted last year but with zero experience). Between now and PASS I am sure I will end up speaking at my own user group for one of the months and maybe get invited to a nearby one to speak. Is that enough?
Should we also place a years of experience guideline? I certainly don’t want to be one of the speakers that people are dissapointed they wasted a session on. But just as much, I certainly don’t want to not share some true level 100 content that I feel is lacking some years if it can help a New-Jr/Jr.-Mid level DBA.
I guess all that rambling tells me I would like to see a SQL Toastmasters kind of program. User groups ad SQL Saturdays with carrots on sticks to solicit good, honest feedback. That is what I am craving. Especially the constructive criticism so I can do better each time.
Maybe it’s okay to put something like below in the program guide at the Summit in the abstract. “Mike Walsh, in his first PASS presentation” Or even go on the premise of badges that I saw on twitter or Allen’s blog comments and have a “New PASS Speaker”.. Not new speaker, period, but new to PASS.. I don’t know.. It’s Friday perhaps I am running out of good ideas waiting for the day to end and a transaction to rollback.
Without a second thought people who speak at PASS should have good experience. I think I remember the summit when lot of newbies took the stage at PASS too and many of us were left wondering how they go tthere. I have my reservations though on chapters and sql saturdays for total freshers, i dont deny there has to be some room for that. But atleast as far as chapters go, in my chapter it turns most people away to have a total fresher up on stage. They think it is a waste of their time, and yeah, they think that can be practiced at well..toastmasters. Yes i have had sp
sorry i mean i have had speakers who are very strong technically and not so great at speaking on stage, those do fine. But those who stammer with technical questions typically do not fare well even with a chapter group, that has been my experience.
I chose to make my community speaking debut at SQL Saturday for a simple reason: If I end up stinking up the place the event is not a total loss.
A trip to the local user group meeting is at least an hour each way for me. My total investment in time quickly gets up around 4 hours. From my perspective that makes the stakes too high to ask people to go all in on my first presentation.
I see SQL Saturday as the get your feet wet in the community event. Even if a couple of presenters stink up the place it is still a kick ass event because it was free.
My suggestion to help the local chapters get more speakers and to help with the vetting of speakers is that your presentation must be nominated by a local PASS chapter to be presented at the national meeting.
Good point.
I must say that i was a bit surprised of the way PASS treat this issue comparing to other conferences, for example Microsoft:
I speak in MS conferences from time to time, and they are always very strict about who is speaking, how good they are, and they even require that you present your session or part of it in front of one of their team.
When I spoke at Pass i felt like there was no real supervision after the first phase of selecting the sessions. Before the conference, no one talked to me to see how my presentation is done or hear part of it. My average was above 4, so I’m not saying they made the wrong choice selecting me :), just stating out that i’m not sure if there’s actually enough done to get the best speakers for PASS…
You know what my only problem is with rating speakers? Likert scales are pretty much worthless without exquisitely designed questions and well-educated, aware, respondents. Even, then most research is showing that Likert scales are bunk because users are most likely to pick 1 or 5 – http://techcrunch.com/2009/09/22/youtube-comes-to-a-5-star-realization-its-ratings-are-useless/
YouTube and Netflix face this problem all the time. Netflix have gotten around it by giving you an indicator of your alleged response (1 – hated, 2 – didn’t like, 3 – like, 4 – really like, 5 – want to make out with). NBC in philly are switching to a “how does this make me feel” response system: http://www.nbcphiladelphia.com/news/local-beat/Worlds-Tallest-Building-Vs-Comcast-Center-80642637.html
If you really want to read a lot of academic blah blah about it, you can visit this delightful blog: http://academicmarketingresearch.wordpress.com/2008/08/05/likert-scale-problems-and-suggested-solutions/
Unfortunately, polar scales produce the type of scores that help people determine whether or not the speaker is amazing. These polar scales actually pose questions that help the speaker gauge how useful their presentation was to the audience. I’d love to make the switch for my user group but I fear that many speakers would be unhappy simply because they don’t get scores that they can use to apply to other conferences.
I’d have to say that I was mighty thankful that the SQL Saturdays were willing to give first time speakers a chance. I think I have done pretty well now. I have spoken at 6 of them, with two more in the next few months (Jacksonville and Pensacola). Without Andy allowing first time speakers, I’m not sure I would have done it. There is no SQL User Group in my community, and most of the user groups outside of my area (Tallahassee, FL) are multiple hours away (Pensacola, Tampa, Jax, Orlando). That really is not too appealing for a work night appearance.
As another item, I have now attended seven events, and only one or two of the sessions I have sat in on were painful to watch. In reality, I have seen more painful sessions at the PASS conferences the past two years than I did at any of the SQL Saturdays.
Of course, I would add my voice to those that say that the speakers at the PASS conference should have some experience. However, I’m not sure how much or what would be considered.