Episode Transcript
[00:00:01] Speaker A: Pilot project versus proof of concept for software implementation hello. Thank you for joining us. This is what counts. A podcast created by Trailblazer Consulting. Here we highlight proven solutions developed through our experience working with companies across various industries. We talk about how you can apply these solutions to your company. We share our experience solving information management challenges like creating and implementing a records retention schedule, creating asset data hierarchy, or helping with email management. This is Lee, and in this episode, Maura and I will discuss the difference between a pilot and a proof of concept when it comes to software implementation.
Maura, I know you have some unique ideas on these things.
Please enlighten us.
[00:00:46] Speaker B: I feel like you put a little spin on that, uniquely.
[00:00:50] Speaker A: Oh, I did.
[00:00:53] Speaker B: So, pilot versus proof of concept. These are phrases that people use interchangeably and somewhat loosely in a lot of ways. But from my perspective, they do have distinct meanings and purposes, and they offer different value to your project. So, proof of concept, thinking about those english words, what they mean is you're trying to test out something and say, is this going to work the way we think it's going to work? And to me, that question leads you to if it doesn't work the way we think it's going to work, then at the end of this proof of concept period, we make a different decision. We're trying to learn something from here, and we mean it. Like, we go into it with the idea that this is our approach, we think this is the right answer, we're going to test it out, but we're open to the possibility that we're wrong and we have to change.
Pilot, on the other hand, is we're very confident this is what we want to do, and it's a bigger project. We know that we need to roll it out in stages, that rolling it out might be disruptive to the business if we try to do it across the whole company at once. So we're going to pilot our software, our approach with a smaller group, and then we might make little tweaks after the pilot before we go to the next phase of rollout. But we're really not open to the idea of changing direction completely at the end of it. So those to me, are the two biggest differences that people don't always pay attention to. But I try, when we are doing this and helping our clients do this, we do try and be careful about it.
The other thing that people sometimes say pilot and what they mean is, like soft launch, is they're going to start small and move to the bigger step later. But they're actually not going to do any lessons learned or any evaluation and analysis and they're not going to make any changes. And that has, in my mind, limited value. So proof of concept, I've got a really good example for this, actually.
[00:03:04] Speaker A: Do either of these items have a technical component to them or do they really hone in on process and people only, or is it all three in each?
[00:03:20] Speaker B: No, they definitely have a technical component, especially in the proof of concept phase. I think when you are doing a proof of concept, it's usually because you have questions about the technology, you have some choices and you've made decisions and they're good decisions and you have reasons for them, but the other choices were also good and that is actually what my example is about.
[00:03:42] Speaker A: So technical feasibility is proof of concept?
[00:03:45] Speaker B: Yes.
[00:03:46] Speaker A: Okay, cool.
[00:03:48] Speaker B: And I think in the pilot stage you get a little bit more away from the technology and more toward the user experience and the success of it. So that could be more of a.
[00:03:59] Speaker A: Will the users use it?
[00:04:01] Speaker B: Yes.
[00:04:01] Speaker A: Okay, cool.
[00:04:01] Speaker B: Exactly.
[00:04:02] Speaker A: Run the same page?
[00:04:03] Speaker B: Yeah. Okay, excellent. Even though you thought I was going to be unique. So here's my example. We helped a client implement an email retention manager management system a couple of years ago and there were very high expectations for this system and there was also huge resistance. And so we did a pretty limited set of requirements and selection because when you come down to it, what you need out of an email system is can you move emails out of your live system into someplace where you can better manage them? It's not extensive functionality, but from a user perspective, the ease of use for moving things, and especially even more importantly, the ease of use for finding things, for doing searches is make or break on success.
So we did this limited selection process. We had a requirements, we had a short requirements document. We looked at a few different options and then the IT team at the client tested it out for themselves for about a month and they were like, yeah, this thing works, it's fine, let's roll it out.
But the legal team was the sponsor of this because it was part of the information management project. There was a whole policy question of how long are they going to let people keep emails in their inbox or in a folder inside the live system before they had to move them to the retention folders. So that took a few months to work out on a policy side. But while they were figuring that out, we wanted to do a broader test of the technology to see if the configuration choices that we'd made were going to be good. Were they the right choices?
And what we started with we're coming at this from a records management perspective in a lot of ways. Thinking about the point of email management is to save those emails that are records. Otherwise you want to get rid of email because it takes up space and it causes trouble. But people are extremely protective of their email. They depend on it. Whether they should or not is a different question for a different day, but they do. It's a very emotional thing. Don't take my email away. So we started this proof of concept and we picked a group of friendly people, but from different organizations, including a couple in the legal department, who knew exactly what we were doing and why we were doing it and were all on board until we got them into the proof of concept. And then they started looking at how hard was it going to be to move emails into folders that depended on the record category and how much thought that was going to take. And so we had about 35 people in this proof of concept for a month. And we got so much feedback on how hard it was to do. And these were from it people, from the lawyers, from a couple of other key business people who had been active in making the decision to move in this direction that we changed the whole approach. From a technical standpoint, we changed the whole approach because we decided, hey, look, there's about 110 record categories at this company, and in any given department you probably have three or four that you might use on a regular basis. It's not a lot of choices, but it was too many choices. And everybody, I think there were 36 people in the proof of concept, they all said, absolutely not, this is way too hard. Can't do it.
So, okay, back to the drawing board. How are we going to do this? And then we started looking at, can we do bigger buckets? Can we just do it by department? Okay. That seemed okay. Went back to the proof of concept team and said, hey, what do you all think about this? Well, second round of horror was, that's the only word for it. Wait, other people can see my emails because by department, everything goes into one big folder. And that meant that your whole department could see your emails. Now, that's because these emails are records. They are records of how your business processes get completed, how decisions get made. They are not your personal thoughts, but again, the emotional interaction, the emotional relationship that people have with email, even when they would look at the sample and be like, oh, yes, that's fine, it's okay. If everybody sees that I approved that, we could pay this invoice.
It was a visceral response. Okay, so then we had to second step after that round of feedback, we actually had to map out by vice president who all the people were in their groups that were going to be in their bucket, in their email retention bucket. And we met with every vice president in the company. We gave them their list. We asked them was it okay. We asked them if they needed further breakdowns and then we had to figure out how to do it because we did not want this to be manual. We wanted to use the active directory process in the company so you could map people easily into their folders and somebody didn't have to pay attention. This is a fast growing company with thousands of employees. You can't have someone have to track what folder do you go in. So that went okay. And we also had to build an exception process for when somebody wanted to have a smaller group in a folder. So if you needed a smaller group that had limited access, we had a way for you to do that.
That proof of concept. That first month when we had people testing it turned into two full months of reworking the approach, the technical approach. How were we configuring the system from the folders perspective and how were we leveraging the data in the active directory records for every employee to put them into the right folder so that we could maintain it. So that proof of concept was hugely valuable, as it turned out.
Then we did think about should we do a pilot, should we do a small rollout or should we do it across the whole company? And what we decided was, in this case, no, this needed to be a big bang. Everybody needed to start managing their email immediately, but we also were going to slow roll into it. So we actually had a six month break in process where we gave them first a month where we asked them to look at their PST files and start putting those into folders and then a month where we asked them to look at inbox folders and start putting those in and then a month to go through their inbox, sent mail, deleted mail. Because of course up until now people have been able to just keep everything they wanted. And there was a time when they were not only allowed to have PST files, but they were encouraged to have PST files because the online system was not working properly. It was not robust enough. If you look back six or eight years in the world of outlook, there was a time when if your inbox was too big, things didn't work and everybody was pushing to PST files for offline storage.
So there was a lot of backlog. So we did that and we also ran training constantly during this six month period. This is before we started actually rolling off any emails. That policy question had been decided while we were going through the proof of concept and these reconfigurations to get them to a six month roll off, which is still very generous compared to some companies, but a lot of pushback. We ended up running ten company wide trainings going through each step of the process. Clean up the psTs, clean up the inbox, clean up the folders, get ready for the roll off, how to manage the roll off, how to be aware that roll off was coming ten of those company wide and another 35 for small groups, individual departments, or smaller groups within a department, as well as a constant string of questions coming into the service desk or getting phone calls. So the entire six months was about the people side of it. So that's where you got to. Our pilot wasn't small group to big group, but it was kind of phasing in getting people ready for yes, your email is going to start rolling off. So the proof of concept really helped us on the way. The pilot reinforced the need to make this change. And then a year later, we started looking at what were people doing and realized that some groups had totally, totally taken to this idea and were putting a lot of emails into the folders. Some were not. So now we're actually looking at refresher training. We're going for micro training, micro bursts of training, about 15 minutes at a shot to just highlight one thing that people need to remember. So it's a journey. But it all started with that proof of concept.
[00:13:35] Speaker A: That was an excellent example. We can use that over and over again. I mean, you got records management in there, you got pilot, you got proof of concept, you got rollout ways and means that you went through a number of different things there. You got exceptions that you had to deal with retention, security monitoring, and compliance. I mean, there's all kinds of examples that we could use that particular example for. Excellent.
[00:14:01] Speaker B: It was email is at the heart of how people are managing business today and how people are exchanging information today. And so trying to do an email management project gives you all kinds of opportunities to test things out and make things better.
[00:14:16] Speaker A: We're good for today.
[00:14:18] Speaker B: I think that's good for today.
[00:14:20] Speaker A: Okay. So if you have any questions, please send us an email at info at trailblazer us.com or look us up on the web at www. Dot Trailblazer us. Thank you for listening and please tune into our next episode. Also, if you like this episode, please be a champion share it with people in your social media network. As always, we appreciate you the listeners. Special thanks goes to Jason Blake, who created our music.
[00:14:44] Speaker B: Thanks everyone. See you soon.
