For consultants and consultants-to-be: BONUS POST!



And you thought the fun was over…

Whether you are thinking about venturing out on your own or have already started, this series will arm you with advice from seasoned consultants! This post features Dr. Barrington, who provides her insight on common questions consultants (or consultants-to-be) might have. Be sure to check out the first and second part of this series!


Gail Vallance Barrington is a graduate of McGill University (BA) and Carleton University (MA) and holds a Doctorate in Educational Administration from the University of Alberta (1981). She is a Credentialed Evaluator and a certified teacher. In 2014, she was made a Fellow of the Certified Management Consultants of Canada. Since starting her consulting practice in 1985 she has conducted over 130 program evaluation studies in the fields of education, health, and research. Her top-rated book, Consulting Start-up and Management: A Guide for Evaluators and Applied Researchers (SAGE, 2012) continues to be popular. In 2008 she received the Canadian Evaluation Society award for her Contribution to Evaluation in Canada and in 2016 was honoured to receive the American Evaluation Association Alva and Gunnar Myrdal Award for Evaluation Practice. She teaches courses in qualitative research and program evaluation for several universities and provides webinars and workshops on consulting skills.

Gail Barrington closeup 2016

What made you decide to become a consultant?

I began to work on program evaluation contracts while teaching as a sessional instructor at the University of Calgary in the Faculty of Education. For a while I did both. Then a term came along when, due to program reorganization, the courses I was teaching would not be offered. I had to decide what to do, wait for a year with half-time consulting or go at it full time. This was not an easy decision as I had always worked for a school, college, or university, never responsible for my own pay check. Going out on my own was very scary. I can remember sitting in my car looking across a rainy street at the office building I had selected and wondering if I could really work there. I took a big breath, got out of the car and the rest is history. My consulting business opened on November 1, 1985.

Could you share some best practices for remote consultancy?

I often work with clients at a distance. For example, I worked for many years on a national evaluation project for the federal government in Ottawa, Ontario while I lived in Calgary, Alberta. I made a point of holding quarterly meetings in person because I felt that being on site with the client was necessary to get a sense of their work, their issues, and their reaction to what we were doing. Email and Skype are fine but in, the end, it is personal chemistry and partnership which bond a project together. In that project we spent time together informally as well as during our day-long meetings. We had fun skating, sailing, and going out to restaurants. These informal activities strengthened our understanding of each other and we accomplished a remarkable amount of success as a result.

Do you suggest that consultants market themselves as a specialist or a generalist?

I think you need a specialty area and then you can branch out from there. New consultants should focus both on what they like doing best and on what they have received good feedback about. Your specialty area will expand and morph over time but having a clearly defined area of specialization as a foundation is a great way to start.

How do you measure the success of a project?

Success is not measured by the final report. It is really measured by the extent to which positive program change occurs. Evaluation is all about making a difference through informed decision making and of course our role is providing the evidence needed. The real results happen long after the report is finished, and we have gone on our way. Not often enough do we circle back to find out what happened afterwards.

Finally, what advice do you wish someone had given you as a new consultant?

Hang in there and find some good colleagues. When I started my business, I didn’t have any role models and so I had to make it up as I went along. The closest approximation I could find to the independent evaluation consultant was the independent business consultant. As a result, I became a Certified Management Consultant (CMC) so I could have some colleagues to talk to. However, research was not their interest area and so I was still struggling about how to conduct evaluation research and bill for it gracefully. Happily, I found the American Evaluation Association (AEA) Independent Consulting Topical Interest Group (IC TIG) and there at last were business people with a social justice perspective. We continue talking to this day!

Want more Dr. Barrington? Visit her website!

For consultants and consultants-to-be: expert advice (pt. 2)

It’s here — part two! 

Whether you are thinking about venturing out on your own or have already started, this two-part series will arm you with advice from seasoned consultants! This post features Ann K. Emery, who provides her insight on common questions consultants (or consultants-to-be) might have. Check out the first post in this series here.


Ann K. Emery is a speaker, workshop facilitator, and blogger, passionate about “making technical information easier to understand for non-technical audiences.” In other words, she is a dataviz expert! Ann is also a well-known blogger, bringing practical tips to those looking to transform their data into effective stories through the use of data visualization. Without further ado, let’s dig into her tips! 


How did you prepare for running your own consulting firm?

Launching my own consulting firm was a happy accident. I started blogging back in 2012, just for the joy of sharing skills with others, without expecting it to lead anything. And I always enjoyed public speaking and leading workshops. My name got out there. People read something I wrote or saw me speak at a conference. I started getting a few invitations to give talks and redesign the visuals in reports. And then I got a few more invitations. And a few more. At the time, I was working full-time and doing grad school at night. I had limited bandwidth for independent consulting projects. In Spring 2014, I finished grad school and had the time to accept some side projects. I did the math; the projects would actually pay more than my current (good) salary. People started asking when I was going independent. I hadn’t considered going solo. I had planned to stay at my current position for a long time. Over the summer of 2014, I spoke with a dozen of my mentors. I got great advice:

“Don’t even think about quitting your salaried job until you have a years’ worth of household expenses saved—and be willing to lose every penny if you’re not profitable the first year.”

“You’ll work harder than ever, but the work will be more fulfilling than ever.”

In the fall of 2014, I was having dinner with some girlfriends, and mentioned that I might go solo someday. “Well, what are you waiting for?” one asked. I didn’t have a good answer. That next week, I put in my notice.

For those wondering how about the transition to a consultant, did you continue working at a 9-5 job until you became established?

I’ve met two types of consultants: those who find themselves with spare time (job loss, just finished grad school and they’re job hunting, etc.) and those who have already built a reputation for doing great work and have prospective clients banging down their door. The first type struggles to take in work. The second type struggles to turn it down. The second type has no choice. You have to quit your salaried job and start your own company. You work harder than ever, for a while. Then you get better at subcontracting and saying no. You get to choose which type of consultant you want to be. Pull the trigger too early—before you’re established—and you may always struggle to bring in work and pay the bills.

How do you avoid being spread too thin?

I hire smart and talented subcontractors like you!

More importantly, I say no so that I can say yes. I don’t appear on podcasts (I’m visual so an auditory medium has zero appeal). I don’t write guest blog posts (my clients hire me to write blog posts so it doesn’t make business sense to write for free). I don’t work for free (I like to keep a roof over my head). I have to decline projects that aren’t a perfect fit so that I have creative energy to rock the ones that are.

Describe a time when you dealt with a difficult client (or situation). How did you make things work?

I divide my projects into two broad categories: training and design.

In training projects—my keynotes, workshops, webinars, and individual coaching sessions—I haven’t had difficult clients, but I have had inexperienced clients. The contact person has been put in charge of planning the keynote address for their conference for the very first time. I often need to teach them about coordinating with A/V staff, setting up projectors, connecting and testing the microphones, and so on. Planning a talk of this level can be a stressful experience for my clients. They want the logistics to be perfect. I try to walk them through the unknowns and alleviate as much of the stress as possible. I’ve given a billion talks. I’ve seen all sorts of stage setups. Everything that could go wrong has gone wrong—projector lightbulbs burning out mid-talk, fire drills, laptop batteries dying, malfunctioning microphones. I get migraines a few times a year—the kind where your vision and smell are all messed up—so I knew it was only a matter of time until I got a migraine during a speaking engagement. It happened in February. I could only see a sliver of my slides thanks to tunnel vision. Then the smells and nausea started. I gave participants a coffee break, puked in the bathroom, and came back and finished the talk. From the audience’s perspective, it was one of the better workshops I led all year. Good public speaking is more about rolling with the punches than about careful preparation. I let my client know that I’ve experienced every possible projector and microphone hiccup and that the talk will be stellar no matter what. 

In design projects—revamping existing reports or designing the visualizations for client reports from scratch—I haven’t had difficult clients, but I have had difficult timelines. Contracting takes longer than expected because we need a signature from someone who’s on vacation. I’m graphing the data and notice that the numbers don’t make sense and they have to re-run the analyses to fix a few typos. Every consultant I know has been in this situation: Something goes wrong that’s outside of your control, and you’re the one who has to give up your weekend to fix it. We all notice the red flags early on. In the past, I’ve tried to give the project the benefit of the doubt. This project will be different, I lie to myself. Sure, their timeline is tight, but maybe everything will go according to plan this time.

My number one goal in 2018 is to trust my gut instinct and decline the projects with too-tight timelines.

How do you measure the success of a project?

Repeat clients and referrals!

A few months ago, I gave a mediocre workshop—or so I’d thought. I’d pose a discussion question to the group, and people just stared at me with poker faces. I’d tell a joke, and people just stared at me with poker faces. I couldn’t understand why the workshop structure I’d carefully crafted over the years had fallen flat. I left feeling deflated. Over the weekend, I had serious self-doubt, questioning whether I was even in the right career path. Then, on Monday morning, the client emailed me, praising the workshop and saying it was the best they’d ever attended. They invited me to return to their organization for another few days of workshops. I returned, gave another few days of workshops, and left with the same self-doubt. For the second time, nobody responded to my discussion questions or laughed at my jokes. And then—you guessed it—the client emailed me, praised the workshop, and invited me to return. The organization is accustomed to traditional, buttoned-up lecturers. My skin gets thicker each time, so when I return for the third series of workshops, I’ll be prepared to pose non-discussion discussion questions and tell my unfunny jokes.

In design projects, I used to think that a repeat client was a bad thing. If I redesigned the report well the first time, the client should be able to follow my steps and do it themselves the next time, right? But my clients are often pressed for time. Or, they can get the design 90% of the way there, and they need me to nudge the visualizations to the finish line. I’m working on a multi-year project right now. Each year, my role shifts. At first, I was creating the visualizations myself. Later, I was coaching their staff members through the process, making minor adjustments to their drafts, but creating very little myself. Other consultants have warned me against this approach, worried that I’ll teach clients too much and be out of a job. But my instincts keep telling me that training up staff is a net positive. I’ve taught the staff so much during this multi-year project that I wish I could hire them. Literally. I tried to subcontract part of a project to one of the women, but we discovered that our contracting language wouldn’t allow it. We’ll definitely be working together again someday.

Finally, what advice do you wish someone had given you as a new consultant?

I’ve learned from the best: Herb Baum, Tanya Beer, David Bernstein, Dave Bruns, Isaac Castillo, Stephanie Evergreen, Edith Hawkins, Rodney Hopson, Kylie Hutchinson, Helene Jennings, Cole Knaflic, Chris Lysy, Kevin McNamee, Johanna Morariu, Kim Narcisso, Veena Pankaj, Maryfrances Porter, Jon Schwabish, and Trina Willard.  

I adore each of these people for telling me what I need to hear, not what I want to hear. They’ve given me all the personal and professional advice I’ll ever need. There’s nothing I wish I would’ve known earlier—just advice I wish I would’ve followed earlier.

Want more?!

If you’re interested in learning more from Ann, check out her website. You won’t be sorry, and I bet you’ll be adding it to your favorites

“Expert” Photo by Rita Morais on Unsplash

New to Evaluation? Here are tips for plugging in!

As a new professional (or one that has recently pivoted) in evaluation, you might be wondering how to leverage yourself or “plug-in” to the community. The beauty of evaluation is its interdisciplinarity but that can make plugging in a little daunting (but not impossible!). Below are some tips on how to immerse yourself in the field!

Become an American Evaluation Association (AEA) member.

Not only will you be able to attend the yearly conference, you will have more opportunities to become involved than you will be able to sign up for. From professional development to peer-reviewed articles, AEA really does have a great compilation of resources for academics and practitioners.

Attend an AEA conference!

Deven Wisner AEA 2017 Evergreen
Me nerding out with Dr. Stephanie Evergreen at the Eval16

It is one thing to become a member and never go to a conference, but this is one conference I am willing to pay out of my own pocket to attend. If you’re looking to share and learn from others, find a job opportunity, or just network with others, this week-long event is a great investment. I can promise you one thing: the AEA conference is like no other (in a good way).

Deven Wisner AEA 2016 NametagFind your Local Area Affiliate on AEA.

Again, AEA is a great resource, and that isn’t just at an international level. They also support AEA affiliates, which means you can be involved throughout the year. This is a great way to meet evaluators near you, find out about independent work (if you’re into that), and further develop yourself as a professional. If possible, I suggest being part of a committee or the board. You will be stretched more than just being a member. We have all become members of organizations to never actually attend an event (c’mon, I know I’m not the only one).

Deven Wisner AZENet
Some of the great Arizona Evaluation Network board members I get to work with!

Join an AEA Topical Interest Group (TIG).

If you have a certain area (or maybe more than one) within the field of evaluation that strikes your fancy, get more involved through a TIG! You might have the opportunity to write a blog post, rate conference proposals, and/or be part of the yearly meetings (held at the AEA conference). Again, you will meet people with similar interests but with different levels of experience. I’m part of the Data Visualization and Reporting TIG, along with Research on Evaluation.

Refine your elevator speech.

Who are you? What’s evaluation? How do other people entitle what you do? All of these things are important. Be ready to explain what you do to others. Dividing my time between industrial-organizational psychology and evaluation means I’ve had to refine this for all areas of my professional life. My best advice is to think back to those family dinners…how do you explain it? Okay, take that and make it relatable each time you talk about it. UC-Davis has a good resource on this here.

Get on Twitter…oh yeah, I said it.

Evaluators are taking on Twitter and it is AWESOME! This is a quick way to see what the trends are and learn from others. Plus, you get to share your own thoughts and work. As someone who was anti-Twitter for a long time, I get it…you might be apprehensive. Twitter is the way to find little nuggets of information that can often times lead to great finds. So, if you haven’t already, create an account and start following other evaluators (pro tip: find one person you like and check out who they’re following)!


Deven Wisner Twitter

…and there you have it! Did I miss something? Feel free to share what has worked for you.

NEW – Additional tips from Ann K. Emery’s blog…

  1. Conference tips for new evaluators
  2. Newbie essentials
  3. Job hunting

P.S. Click here to read a blog I wrote for AEA365 as a Data Visualization and Reporting TIG member.

Got an applied project? You can build the capacity for data-driven decision making.

During graduate school, students are usually offered applied opportunities. What I love about applied psychology (e.g., evaluation, I-O psychology…) is that graduate students have the chance to bring their knowledge to a variety of industries — and built the value of data driven decision making. To me, that is priceless. Exposing the field of applied psych is great…and so is making others aware of all the great things can be done when something other than anecdotes are the decision making tool of choice.

So what’s my experience with this? I also had these great applied opportunities, and I started to realize that I was an advocate for my field. I had a new perspective about the entire experience — if my client walked away feeling like they wasted their time, I didn’t do a very good job.

My second to last semester I completed a needs assessment and process evaluation for a company in another state. This company is phenomenal — great idea that’s meeting a need, lean bottom line, and an office full of great people. Where’s the but? Well, it’s that data wasn’t driving their strategic planning. Needs assessment? What’s that? There I had it — an opportunity to BUILD the capacity of evaluation in this organization.

In short, the project went well — everyone learned a lot, services were revised, future planning was focused on data. But that’s all while you’re still there, right? In the back of your obsessive applied psych mind…you know this was one project, and the long lived method of luck and “educated” guessing (oxymoron’s make for good blog topics) could be revived and become the preferred decision making tool — again.

Data driven decision making. What does the data tell us? How do the statistics relate to what we are seeing financially? Your clients said they wanted this service…but was it a representative sample?? I felt like a broken record…because I hoped that between demonstration, dialogue, and bringing my client along for the experience would lead to an appreciation and PREFERENCE for data to inform their decisions.

Created by Chris Lysy

Well, two months later I was in a follow up meeting…sitting in on an unrelated project…and I heard it:

“We need to make decisions based on what the data tells us. It can’t be what I like, or what makes sense to me. Let’s use the tools we have to make changes using data.”

…you know that moment when someone says their idea of an awesome day is binge watching Frasier and eating pizza rolls, and you’re like “…that’s hot.” Bam. There it was. The sexy side of being an advocate for our field.

You see, change is hard. Pushing for a better method (that isn’t always easier) can be a challenge. And hey, being a grad student is a special level of hell at times. Sometimes you want to drop the results and peace. You don’t always want to screw with Excel for hours to get something other than a canned report (but seriously, talk to me if it’s taking you hours to craft good viz). But at the end of it, you have an opportunity to see the results put into action. Your very presence is a disruption — a potential catalyst for change. The credibility of our field? It’s on ALL of our shoulders. So, the next time you’re burnt out, remember the potential to impact decision makers and the responsibility to your colleagues (oh, and call on them when you need help!).

Created by Chris Lysy

Perfect datasets don’t exist but cool tools do. Here are two I use.

One component of my day-to-day is retrieving reports and turning them into something useful. Simple, right? Ha! I bet you laughed, too. Data comes in all shapes and sizes…in fact, it usually comes in every form besides the one we need. As data scientists, our job isn’t only to interpret the data and provide someone with the results to make an informed decisions – although that’s great. Visualizations? Those are great, too. But, if you don’t start with the right data, you might as well not analyze or visualize it. …so what?

Well, canned reports suck (unless they’re super customized by your IT team, but who has time for that?!?). When I ask for a data pull, I hate being told “no” or “yes, I’ll get that to you in a few weeks.” Usually when I need a report, it’s because someone important needs to make a decision. So instead of waiting around for someone to do it for me, I put on my data manipulation cap and get ready to finesse some Excel files.

Enough with the back story, right? You need tools! Admittedly, my most common problem is having all the data I need…spread across a bunch of out-of-the-box reports. Easy you say? Sure, if I was only dealing with a few rows of data. We’re talking thousands of rows. So, here are a few tools I’ve come across that have turned my data nightmare into something usable:

  1. Consolidate
  2. Merge Table Wizard for Microsoft Excel (the app with the world’s longest name) – Mac
    1. If you’re a PC user, use ActiveData

Screen Shot 2017-04-17 at 8.35.21 PM

So let’s talk about these tools…

Consolidate is great when you have several entries for one company (or participant) that you need summed, averaged, or a number of other functions. It’s as easy as five steps:

  1. Select your data.
  2. Go to the Data tab.
  3. Choose the Consolidate function.Screen Shot 2017-04-17 at 9.07.41 PM.png
  4. Select your data (be sure to note where the label that you’re consolidating is at).Screen Shot 2017-04-17 at 8.37.35 PM
  5. Choose your function.

Screen Shot 2017-04-17 at 8.38.22 PM

Done! Now you’ve got your data in one place. Screen Shot 2017-04-17 at 9.14.02 PM.pngIn my case, I needed each company’s entry to be consolidated into one entry, and I wanted the sum of all of their orders.
Next, I wanted to tie e-mail addresses to the company. That’s where my fancy application came into play. Here’s a quick walk-through on merging two reports to make one useful report.

  1. Open both reports.
  2. Open Merge Tables Wizard (Mac)
    1. ActiveData (PC)Screen Shot 2017-04-17 at 8.47.35 PM
  3. Select the data you want to merge on (pick the dataset with the majority of the data). This is your primary dataset.
  4. From the other worksheet (or, in my case, a workbook), select the data you want to merge. Screen Shot 2017-04-17 at 8.48.36 PM
  5. Your tool will likely verify that it identified the variable you want to merge on (i.e. company).Screen Shot 2017-04-17 at 8.48.46 PM
  6. The default is to add a new column for any data that does not exist in the primary.
  7. After that, the remaining steps are fluff. Select what you want and run it!

Boom. Screen Shot 2017-04-17 at 8.49.34 PM.pngIn several minutes, you just created a data file with all the data you needed. Of course, this is an oversimplification. But, you can apply it to anything – especially those nasty datasets.

So, what about you? What are your favorite tools for forcing Excel to comply?Screen Shot 2017-04-17 at 9.14.02 PM.png

I stopped blaming Excel for my basic data visualizations. Here’s when and why.

Early last year I realized my research findings only meant something if I involved the right people. And, to do so required me to be a creative and adaptive evaluator. Around the same time, I realized that I could have all the answers, but without a good way to communicate them, no one would actually notice.

Screen Shot 2017-04-02 at 8.32.46 PM.png
Look how easy it is to be basic!
I started searching for solutions. I found blogs by industry celebs (Ann K. Emery, and Stephanie Evergreen, to name a few) and began to pour over their archives. I found a lot of cool stuff. My world was quickly shaken when I realized that meant APA formatting might not be the best way for my voice to be heard (my condolences to the die hards, there is a time and a place). But, the big takeaway was that I figured out that, “Yes, it sucks because Excel made me do it,” was a crappy excuse for not visualizing my data in a better way. I learned to stop accepting the norm. You see, the purpose of Excel is to be a tool – not to do all my work for me because guess what? I’m the expert! I know those gridlines are garbage (and unnecessary), and that I should really figure out the RGB numbers for my brand.

So when did I stop blaming Excel? When I started valuing my data visualizations as more than entering data, selecting insert, choosing between a column or bar chart, and dragging it into my equally as boring report. When I realized that sweet visualizations in Excel just meant more thought and a few extra steps.

Screen Shot 2017-04-02 at 7.13.36 PM
I stopped accepting default settings!
Need an example? Check out Basic to Bomb Column Charts! Now go…find your inspiration for better visuals!

Screen Shot 2017-04-02 at 8.53.49 PM
Not too bad for a few extra clicks…

Basic to bomb column charts.

Screen Shot 2017-04-02 at 8.54.25 PM
The basic Excel column chart.
Let’s take a quick look at how a basic column chart in Excel can be turned into something you might want to look at.

  1. Enter your data into Excel
  2. Insert a column chart (if you’re basic, save and close here…)

Oh, good! You’re ready for more. Okay, here it goes:

  1. Get rid of that white space by decreasing the gap width.
  2. Remove the gridlines. They add unnecessary clutter for readers
  3. Trash the x-axis.
  4. …the y-axis, too!
  5. Add data labels (and for the love of all things, don’t leave them hanging in space – move them onto their respective bar).
  6. The title. I’m a fan of Stephanie Evergreen’s piece on this. Instead of a simple word or two, write a sentence that describes the dataset and LEFT justify it.
  7. Format! 10pt labels, 12pt title, non-default font and colors.

Bonus: Make everything a table. Sort them — highest to lowest (i.e. descending first), so readers don’t have to do the ordering themselves. 

Questions? Shoot me a message or tweet. 

Screen Shot 2017-04-02 at 8.53.49 PM
Our column chart gone bomb!