Virtual Conference Attendance Worked: I was Amazed at How Much I Learned!

I suddenly found myself able to attend the American Evaluation Association (AEA) annual conference virtually because I had a relatively open schedule during the conference days of Oct. 31-November 3, 2018 (Wednesday afternoon-Saturday afternoon). I didn’t know until close to the conference time that AEA would have virtual access – and not just to the Opening Plenary.

Thanks, AEA, for the free and excellent streaming access to Presidential Strand sessions!

So I began attending the virtual sessions. And I was deliberate and diligent in engaging with the conference Twitter hashtag #Eval18 (and other variations).

By the 2^nd day afternoon, I realized what a full picture I was getting of the conference even though I wasn’t there. Attending tweeters were keeping us updated on all aspects of the conference. And I found myself reading threads and going back to the program abstracts to learn more about sessions that weren’t streamed but whose content impressed an attendee enough to write a tweet.

And I also began to aggregate in my mind (an “asset” of being an evaluator) the learning I was getting from the conference. So, I went back to my decade-plus as a training and performance evaluator to formalize this learning and evaluate it. The intended audience for this retrospective research was a sample of 1 virtual attendee [me]. I used the Kirkpatrick Four-Level Training Evaluation Model to judge my “learning” and subsequent application from the conference.

Data Sources

·        AEA Presidential Strand (Virtual) Sessions

I was able to attend 5 of the 7 virtual sessions streamed and 3 of the 4 Presidential Strand sessions during the evaluation focus period.

·        Twitter

I also engaged on Twitter tweeting my observations and learning and following #Eval18.

For the Twitter analysis, I used all Tweets with #Eval18 from Wed Oct 31, 2018, 00:02:16 GMT (4 hours ahead of Cleveland) to November 1, 2018, 21:51PM. [I decided to start collecting data on Friday to document my learning. I restricted the data collection to the first 2 conference days (Wednesday and Thursday) so I wouldn't alter and bias my Twitter and AEA virtual-attendance behavior once I decided to collect the data.  😊]

I found a free program (easy to use quickly and get data out of in a Google Sheet format from TAGS V.6.1.9 (https://tags.hawksey.info/get-tags/) to extract the 1944 tweets.

I excluded nice-to-have tweets.

This left 413 tweets that I categorized and did content analysis on in Excel. 

Results

Exemplar tweets for all the categories are woven into the results along with observations from virtual attendees (primarily me).

• Great reminders

 I labelled these as Indigenous tweets and they came from Nicole Bowman and others, setting the stage for those arriving in Cleveland to keep in mind the long history we evaluators need to bring to our work – and very relevant to the conference theme of “Speaking Truth to Power.”

• Getting To and Waiting for Sessions to Start

Virtual attendees were always greeted by lovely (not loud) music and these informative and changing slides between presentations. IRL attendees (In Real Life, IRLs going forward) for streamed sessions had the same slides. (No one tweet-commented on whether music was being played with the slides the IRL attendees were watching.)

• Healthful Practices During the Conference

Virtual attendees (or at least me) got no exercise during the conference, but I did have my choice of seats and healthy snacks and meals. IRLs got lots of steps in and/or used the workout and other healthful areas. And IRLs had Food Trucks – and added even more steps to reach them.

• Virtual attendees had as good or better views of both presenters and slides – at least in plenaries.

Virtual attendee views of the presenters and slides were excellent – and didn’t depend on the room size or the number of chairs.

• Virtual attendees could ask questions “equally” with IRLs

Each virtual session had an active chat window that helped us see who else was attending as well as for commenting, networking, or asking a question. Presenters would answer the virtual questions equally with attendee questions. The AEA virtual moderator asked about people's locations outside the US. And some virtual attendees tweeted about their engagement like the evaluator from Tokyo. 

• Most of the Tweets were to share value-added content.

77% of the tweets were specifically session content – value-added information that couldn’t be gleaned from the session titles. More than half of the tweets (218) were specific content shared and an additional 100 were observations/comments or tips and tools.

a.      Tweeters shared a range of content from many sessions I may never have considered attending if I’d been an IRL attendee. Specific content such as pictures of slides being presented were very useful as presentation handouts are much less available (according to 1 tweeter) and are less timely since they aren’t usually available til after the conference. I was really amazed at how many new ways to approach evaluation are being used and how adaptations are working out in different situations.

b.      Tweets on tools and tips were very helpful although they rarely came with links to find more information for following up.

c.      Additionally, tweets with comments and questions were helpful perspectives to remind us of taking our learning forward.

• Other Kinds of Information and Commentary

a.      Both IRLs and other tweeters shared information useful to evaluators that amplified or supported the conference theme.

b.      There were fun tweets – and not just around the Oct. 31rst conference day. And an instant classic from Chris Lysy whose ability to translate evaluation into a comic is unbelievable.

c.      I was impressed that 2 “I’m-a-first-timer” tweets included content as well.

  

d.  Tweeters pointedly gave feedback for AEA and TIGs.

e.      And a tweet about Michael Scriven’s absence was acknowledged through his good wishes to all of us for a good conference was shared (and RTed repeatedly).

Summary

Twitter was the critical link to expand my virtual conference attendance into more of a full conference experience. There were dedicated live tweeters like Zach Tilton (recognized below in a tweet from Sara Vaca a couple days ago as he has continued to share threads of his live-tweeting), Allison Titcomb, Jara Dean-Coffey, and Tom Archibald. So many tweeters provided insights into and pictures on session content that gave me learning from and insights into evaluation topics I didn’t know about.

But the variety of tweets I reviewed showed a range of conference aspects. These all gave me real feelings of being IRL with the 3000 IRLs in Cleveland since I could see activities beyond sessions and posters, learn about and marvel at the range of health and wellness facilities and the Food Trucks, see groups’ meet-ups and fun times (especially those making a “work” connection to their fun times), and enjoy pics of people meeting IRL after being connected electronically (including “I finally met my hero ___”).

I also did much more engaged networking, primarily through the live-tweeting I did. This resulted in many more Twitter followers by the end of the conference which has greatly increased my Evaluators PhD and Related Twitter list, one of the first lists I peruse when I am on Twitter.

And I better solidified my learning by doing this reflection and review. To ensure that I applied the learning into action in the short term rather than leaving the conference with so much information and not being intentional about using it, I committed to develop it into this blogpost. When I tweeted about this as #Eval18Results, several others also contributed their commitments.

(DM me on Twitter or email via SueMentors on gmail if you would like access to the data.)