December 2020

Image source Shutterstock.com. Free for commercial use, no attribution required.

Question: Have you ever wondered how many types of analytic insights you can get out of one data set?

Answer: Keep reading to learn the answer.

​

First, let's try an experiment. Close your eyes (note, you may want to read all the instructions before you do this…) and picture a curvy road. Take 15 or so seconds to commit the details of your vision to memory. Now, open your eyes. Did you picture the road in the country or desert or mountains? Was your view from above, or were you on the road? How much of the road could you see, and were there any hazards ahead? What was the weather, and was it day or night?

I will bet my entire stash of Bitcoin that you can't guess what I pictured when I wrote this, nor could I guess what you pictured just now. Neither of us is a mind reader.

Consider the following scenario. A division chief - we'll call her Ms. Q - walks into the space where her data science team sits. The team is a group of very bright and curious professionals including a data scientist, data analyst, analytic developer and statistician, who are all eager for a new task. Ms. Q states that another division chief (Mr. K) has requested that the team analyze a newly acquired dataset to 'see what they can find.' Ms. Q shows the team where the dataset is stored on the network, then walks out to her next meeting.

The team is looking forward to the prospect of providing real operational value, and they throw everything they have at the task. Along the way they hit a few dead ends, but after a few weeks they think they have something of interest to show. Ms. Q and Mr. K walk into the space to view the results, and the team shows them a map with clusters of dots. The team is super excited, and begins describing all the highly technical work that went into the map and what they think the clusters might mean.

Ms. Q is just happy to see some results, as she has been getting impatient for the past week or so. She smiles and says 'Wow, that does look pretty interesting. Mr. K, what do you think?" To which Mr. K replies "Hmm, I'll have to take this back to my analysts to let them look at it, I'm not sure exactly if this is what they were expecting. But thank you for all your hard work." The data science team never hears back from Mr. K, and has no idea whether or not they provided him something he valued.

Now, answer the following question honestly. As you were reading this, were you clear on what Mr. K's requirements were, or if he found the results valuable? Me neither, and I wrote the scenario. Queue the mind reader.

Applying an Agile mindset to the project could have produced a better outcome. Agile provides approaches for facilitating the work in a way that clarifies requirements, sets expectations and delivers real value without stifling the exploratory nature of data science. In fact, Agile methods are ideal for projects that have a high degree of uncertainty. 

In a modified scenario, the Agile Product Owner worked the requirements out with Mr. K, and translated them into a prioritized backlog of user stories. The entire team then planned how the stories would be completed, and estimated the level of effort required; committing to a plan that they created as opposed to one dictated by others. The self-organizing and motivated team was led by a Servant-Leader (e.g. Scrum Master) who empowered and facilitated their work. The Product Owner supported by providing the team with the information they needed to stay on track to meet Mr. K's expectations. The team met frequently with Mr. K and Ms. Q to review progress and requirements. 

Using an Agile mindset, the team delivered real operational value to Mr. K early and continuously. Expectations of time and delivery were clearly set and either met or adjusted with full transparency and understanding. The team was able to decide how the prioritized work would be accomplished without being micromanaged. And Ms. Q is pretty happy. Boom!  

This scenario is entirely possible. Data science teams inherently employ a lot of Agile practices already, like collocation, osmotic communication, self-organization, swarming and even DevSecOps. Now, I have seen some articles resisting the application of Agile to data science projects. I believe that data scientists resist because they feel that Agile will be too restrictive, and leaders resist because they believe that there is not enough planning in Agile. To both of you, I say the following: 

• Data scientists: Agile provides you a Product Owner who manages the customer and clarifies requirements, the freedom to choose your tools and determine how you will accomplish your tasks, a Servant-Leader who exists to facilitate your success and handles the requests that fly in from every direction, the empowerment to lead and explore, and a safe environment where you and your team can communicate openly. Yes, there is a time estimate to follow, but you set and commit to the estimate at the start of the period based on your expertise.

• Leaders: Agile provides you a data science team that is motivated and constantly improving, a product owner dedicated to understanding your requirements, the ability to change the team's direction as needed with minimal loss of efficiency, the assurance through regular demos that the final product will be what you value, and information radiators that give you continuous insight into the team's progress. No, there is no Gantt chart, but you do get an estimate that is likely better informed and flexible enough to change as your requirements do.

What's not to like about all that? I rest my case.

Now, back to that question about how many types of analytic insights are possible from a single dataset. Would the answer provide you true value, or did the question itself provide the value? In any case, I'm guessing that number is very, very large, and the likelihood that Mr. K and the data science team had the same type of results in mind is nearly zero. 

Please reach out to Cybele Data Advisory if you would like assistance with empowering your data scientists and firing your mind reader.