Acronym

Scott -
In my own experience planning meetings and working with speakers, as well as attending, I often find that speakers promise the world - usually through a broad, sometimes vague, description. When that information is requested so far in advance of actual preparation of the presentation, I think it's inevitable there will be a difference between what's promised and what's delivered. Hopefully, what's delivered is better, but not always, as you have noted.

And there is always the possibility that between the time the abstract is submitted and the presentation is given, something changes significantly in the marketplace to warrant a change in direction for the presentation.

Of course, as a planner, that kind of lead time is needed to market the meeting. So, it's quite the conundrum!

I haven't given any answers, but I think that is exactly what has happened. I had a similar experience at ASAE. I think you are correct about lack of time being the culprit, but I am hesitant to blame attendees (unless, of course, you have specific evidence to do so). I look at titles and speakers, and the value of the experience in the past, and decide whether it is worth my investment. (If I haven't attended before, I look at the sessions more closely.) It might be worth looking at cute and "hyped up" titles to see if those could be rewritten to more accurately represent the content of the session.

Scott,
I think you have your finger on something - time. But it's not the attendees' fault. With PD budgets trimmed so much, we really want every session we attend to be useful. So when a session is bad, we're not just disappointed over wasting an hour of our time; we feel ripped off. Our expectations and our hopes are higher because we have so little time for PD.

I think Kristi has hit on the key theme. Session descriptions are brief, so speakers tend to make broad statements, and many of the learning outcomes provided are generic or too vague. Yet when proposing (including for ASAE & the Center) we are asked to provide much longer descriptions than what is then used for marketing.

We need to think more about what information would help participants make the most informed choices and then figure out how that information can be provided. At minimum, online listings should contain expanded descriptions and/or tentative session outlines.

Session proposals and session marketing haven't changed much in the past decade. It's about time we rethink whether or not the current processes are still valuable.

Scott, I actually we get pretty GOOD reviews and certainly, numerically they are. But the difference is that COA programs (or anything that is tightly focused) also has a smaller, homogeneous audience. It really helps to know who (specifically and generally) you are going to be speaking to, and meeting planners do us no favor when they don't share the attendee list in advance. Of course, in a big conference one can do all the sharing in the world and it doesn't help because the event will be full of Kristis and Kevins who are shopping up to the last minute and then some.

To Frank's point, the way I calculate paid attendance is for example with a $500 registration and $600 related travel costs (i.e. Southwest and Priceline!) with two keynotes and eight total concurrent session time slots, my PD purchase decision just cost me $110 per session. Would I be upset if I bought tickets to a Broadway musical and today's stand-in for King & I was a dancing gorilla? Of course I would be! But theatre protects its reputation for quality better than we do, and supports a small industry of reviewers who can also expertly measure and communicate program quality. In conferences, we generally have no similar objective party and we have to rely on a limited range of historical audience scores provided by 1/4 to 1/2 of those who attended for the duration.

I fear that sometimes speaking, or coordinating content for conferences can both be semi-no win situations. Audiences today are so diverse in terms of their experiences and background. A speaker has to be very careful articulating lessons learned from case studies of campaigns or specific associations. It helps to mix basic and more sophisticated insights to ensure you're meeting some of the needs of people with a wide range of responsibilities, attitudes, past experiences, even preferred learning styles. I only get 4.5s from COA programs today and feel lucky to get 4.2s from some DMA or ASAE programs; other programs, such as panels aren't as good and I share the responsibility if our scores are lower. I guess there is no silver bullet, and it's more noticeable at a time when business travel cuts and time constraints all make us more demanding in the audience; life doesn't necessarily get easier at the podium or for the planners.

Jeffrey's comment about learning outcomes being vague is on the money. If speakers could commit to one-to-three tangible, actionable, doable outcomes (and be able to deliver on them), it would go a long way toward making attendee expectations meet a session's reality.

It wouldn't work for all types of sessions, but for the more how-to sessions, I think it would eliminate a lot of problems.

This is a topic near and dear to me...I agree with all that is said, but I think there are opportunities on the analysis side to dig in a bit, and work harder on two things: 1) figuring out what the pains/needs are 2) Creating edu events that fill those gaps---

I'm amazed at how many ways there are to analyze marketing and financials, and set up strategic plans and action steps to achieve success for them...but when it comes to education, I know our association has been guilty of simply a lack of in-depth analysis and review--if you can measure it you can manage it...over the past few years, we have been digging into it a bit, here are some things I'm working on, I would love some feedback as I am sort of shooting from the hip on a lot of this stuff! We also drive metrics now related and cross-referencing attendee feedback, type of training (online/self/in person), and financial success

1) Relevance: I am working on developing some metrics that help measure relevance of a topic based on I think these two factors; A) Attendance---how many people attended (may have to be weighted depending on the event) and B) What were the averages from their ratings?

2) Establish a baseline: 4.1 may be good or horrible, depending on the scale...and the audience---we have worked on averaging the scores of all of our events, in teh attempt to create a baseline--then, in our review process, we can use that baseline as another way to evaluate a session---if it was above the baseline, why? If it was below it, why?

3) Prioritize and Strategize---this might be much more difficult for larger associations and industries, but each year we go through an intense analysis phase that takes a few months...we go through all of our topic proposals, look at surveys, and talk to our Edu committee...out of that, we develop an educational agenda that we then work on prioritizing through our board outcomes and strat plan and from the current challenges---the agenda is the starting point for us to develop our training for the next year...this process is much harder than simply asking for proposals and sort of putting sessions in to simple topics like 'Business' etc.

4) Scope: I am always nervous when I see that we are planning an 1.5 hour session on 'Marketing'...granted some big picture courses can and are needed, but narrowing the scope helps---this will most likely need to be managed by staff, although highly professoinal speakers should have this down

Again, just brain dumping, would love to hear from others, and I'd be happy to share what we are doing with anyone.

A small amount of words (100) to describe a long amount of time (75 minutes) means the speakers must leave much to the attendees' imagination, hence the expectations that aren't met. So tell me, in this age of the internet, why on earth are we limiting the session descriptions so much?

Two thoughts here: Jeffrey and Sue have mentioned the need for clear learning outcomes. Here's a good way that might make the understanding of those outcomes crystal clear for both speakers and attendees: make the list of learning outcomes the published session description. Just that, nothing more. Phrased as "attendees WILL learn...". If speakers know that these statements will be the sole basis upon which attendees choose their session, they'll be far more inclined to make their stated outcomes tangible and realistic and strive to meet those exact goals in their sessions.

To Jamie's question about limiting session descriptions, the internet age has given us all an interesting conundrum: space is limitless, so information can be longer, but because of information overload, readers skim more and read less. So, offering longer session descriptions online can't hurt (other than added administrative time for publishing them), but I doubt that a lot of attendees would take the extra time for extra reading (some would, but not a lot, in my estimate).

I'm reminded of the Mark Twain quote "I didn't have time to write a short letter, so I wrote a long one instead." I'd personally prefer the same short session descriptions, just more clearly written (perhaps a la the learning outcomes mentioned above). Unfortunately, that takes a lot of work. It's a tough challenge for any meeting planner or marketing department. Writing one clear session description is hard enough, let alone writing 100 for a large conference.

How about the longer the session the more info we should publish? I think you're right Joe about a smaller percentage of folks digging in deep for a block of 75-minute concurrent sessions, but for half-day and full-day sessions, I want a lot more info. Compare what is sued to promote a freestanding full-day program with a full-day at an annual and there is a dramatic difference.

I think Brian's effort to assess relevance is a key commitment, so why don't we ask it on the eval form: What percentage of this session's content was relevant for you? Or make it open-ended: what was most relevant and helpful? what wasn't relevant or helpful?

One of the problems of going to all online evals after the conference is we lose some of the insight people can offer when evaluating a session just after it's concluded. I've noticed the # of evals. I receive and the quality of the comments has declined significantly as we've moved from paper to online evals.

Finally, let's acknowledge the not-too-secret reality that lots of folks write a rather generic description with vague outcomes because they are proposing a session months in advance and are often doing it at the last minute. Many haven't even thought about the session design, let alone outlined the content sufficiently to produce a description and outcomes that would be more focused.

So many of the parts of this process need to be rethought.

And just to really shake things up, let's start publishing speaker evaluation averages from the last session(s) they did along with making online comments (like epinions, university prof evals, Amazon reviews, etc.) available.

I continually struggle with this question and I echo many of the sentiments both in the original post and the subsequent comments.

We recently redesigned our session evaluations to attempt to address some of these issues and separate "content" from "speaker." Sometimes great speakers deliver crappy content and the other way around. This is also helping us better dial in panels which are admittedly tricky. Oftentimes one bad panelist can skew the entire session rating down when in all actuality, the audience may actually have found quite a bit of valuable content and positive perceptions of other panelists. If you can get them to separate those things away from their desire to blast away at the one lousy panelist you have a better chance at getting a more nuanced picture of what actually happened.

We are also trying to assess both "years in association management" and "years in current staff level position" which are sometimes vastly different answers and can't help but influence an audience members perception.

By moving to online session evaluations we have increased our response rate while at the same time compressing the bell curve. It seems to ameliorate the outliers, both positive and negative. Some folks get so inspired in a session they rate all 5s just because they are excited. Some folks just got up on the wrong side of the bed and decide to be a hater for no good reason. Neither one of those constituencies seem to show up en masse a few days later when some amount of calmer reflection time has set in.

I really do not have any answers. I wonder what will happen as we move more and more towards interactive experiences, which by their very nature, make it even harder to predict what will actually happen in the room?

Great discussion. Thanks for starting it.

Shelly

I like Joe's suggestion of publishing the learning outcomes that speakers are required to submit. Those may not be stated as attractively as marketing copy would be, but maybe that's the point.

Tell attendees what they'll get, now how great you think they'll feel after they get whatever it is they are given.

What a great discussion! I had no idea this was such a hot-button issue.

I think Jeffrey really put his finger on the main reason for the disconnect between session descriptions and the actual session itself: "Finally, let's acknowledge the not-too-secret reality that lots of folks write a rather generic description with vague outcomes because they are proposing a session months in advance and are often doing it at the last minute. Many haven't even thought about the session design, let alone outlined the content sufficiently to produce a description and outcomes that would be more focused."

Another problem I've seen more often recently is speakers dropping out more regularly due to budget cuts or travel restrictions being placed after proposals are due but before the conference takes place. Changes in speakers can have a huge impact on what an audience actually gets when they show up for a session, especially if the speaker is recruited very late in the game and is coming up with some content very quickly.

Perhaps more interim deadlines for speakers would help, to make sure they're on track with preparing their content and also to make sure that session descriptions are accurate and informative. But more interim deadlines can also lead to unhappy volunteers, if they feel overburdened by the requirements.

Are there any conferences you've been to that you feel do a great job of matching session descriptions with content? Maybe some of what they're doing right can be helpful advice for other conference planners.

Hi everyone,
I was definitely not blaming attendees. In some ways I was not blaming anyone. I was just trying to point out the challenges we all face right now when trying to get the right content to the right people at the right time so that attendees get a good return on their investment and the organizing association benefits through an enhanced reputation, membership growth and increased revenue. This is becoming more and more difficult and I think we all understand this but as Lisa commented the challenge seems to have lived under the radar. I want it to come to the surface so that we figure out how to do things better so that all of the goals mentioned earlier are achieved. Any ideas on how to do it?

Great discussion here.

As an event professional and conference organizer, I take the workshop descriptions and learner outcomes very seriously. If you know me, you know that I view designing a conference through the lens of an educator and believe education design is often overlooked by event professionals in lieu of conference logistics.

I believe the session and workshop descriptions are extremely important. I use the following phrase in our session descriptions for the learner objectives (LOs): "After attending this session, the participant will be able to..." Then I list three to five LOs. (As an educator, we are trained to write quantifiable LOs that can be scored and evaluated easily. You can’t evaluate if someone improved his or her sales skills after a presentation. You can evaluate if they learned seven ways to improve their sales skills.)

Then I make the LOs quantifiable using Bloom's Taxonomy of Higher Level thinking skills. (Presenters should learn these six levels of cognitive domain. Try here http://www.officeport.com/edu/blooms.htm and http://uwf.edu/cutla/assessstudent.cfm to start.

I insist that the LOs use action verbs that can be evaluated such as "Identify seven ways..." "Compare and contrast..." etc. I work with my speakers to help them understand this process and why it is important. Then I urge each speaker to start their presentation with the LOs as part of the agenda and to emphasize when they cover each LO. I suggest that they end with the LOs again and restate them. We learn best with repetition. That way the attendee has a clear way to see when each LO is covered and a recap of them. On my evaluations, we ask the attendee if the speaker met the LOs.

If more conference organizers and presenters ensured that the session descriptions were accurate, that their LOs provided value, and that the presenter delivered on their word, we see more satisfied attendees. (At least we would see attendees say that they description accurately described the presentation. Whether the presentation met the attendees’ needs is a different story.)