Stop Scope Creep: Must-Ask Questions for Data Analysis Projects

Data analytics projects can end up in an endless loop of revisions and missed deadlines like any other project. One minute you’re committing to a deadline that sounds easy, and the next minute you find yourself trapped in a seemingly never-ending loop of revisions, extensions, and “Just one more thing” requests. This is true for a simple one-time analysis or an ongoing dashboard.

While I don’t think you’ll ever get to a point of ZERO changes, you can definitely pre-empt a lot of avoidable ones by thinking through the questions to ask for data analysis projects BEFORE you get started with a single query.

Whether you’re a seasoned data pro or aspiring one, taking the time to ask pointed, purposeful questions prior to starting any data project can save you not only time but the sanity-fraying experience of multiple changes that will feel like fire drills.

Form, Email or Meeting: How I Like to Gather Information

I’ll start by recognizing that each team and organization is so completely different. When working with a team that is structured with a well-established data team, an intake form usually works just fine. At a company that doesn’t have well-defined data warehouses, the form won’t fly. Those customers will likely benefit from conversations for a couple of reasons:

• They think they need 1 thing but really need another
• They don’t know what they don’t know

Where the teams involved & the organization are in data maturity will play into this from unpredictable & reactive to measured & controlled <See SSA Analytics CoE Advanced Analytics Capability Maturity Model presentation>

Regardless, my ideal workflow is to intake a request through a form. It’s standardized & adds enough friction that requires someone to think through their request & hopefully get enough detail to limit the back & forth to deliver what is needed.

Ultimately, a basic request could get prioritized and worked on if enough details are provided without reaching out, but bigger projects would still likely get a meeting even if they came in with great detail. The questions we’ll walk through here can be a consideration whether you have a standardized form or need to have questions on hand on-the-fly for requests, too. The point here is to get a full picture of what the ideal outcome is if this person/team had the end state request in their hand.

What they think the problem is that they think data can solve isn’t always going to get to the heart of the problem. Better questions or better conversations is how that is going to happen.

Here’s an example form.

Project Blueprint: Stakeholder Goals & Expectations

The first step in our systematic approach to any data project begins with capturing clear goals and expectations. These questions to ask for data analysis success aren’t merely a box-ticking exercise; it’s the cornerstone of a successful project. We’re aiming to understand not just what you want, but why you want it and how it aligns with broader objectives or key performance indicators (KPIs).

There are two key elements here: What stakeholders want to achieve (Goals) and what they expect to happen (Expectations).

Goals are generally the endgame, the ultimate achievements you’re looking for. They need to be specific, measurable, achievable, relevant, and time-bound (SMART). For instance, ‘We want to increase monthly user engagement by 20% within the next quarter’ is a well-defined goal. Some organizations have gone so far as tying a monetary value to a project to make sure resources are being spent well.

Expectations, on the other hand, are the assumptions or beliefs about the project or the process, often tied to goals but more nuanced. An expectation might be, ‘We expect to identify bottlenecks in user engagement through this analysis.’

At this stage, if you’re using a form, there should be fields that help capture both these elements. The aim is to be as clear and specific as possible so that the entire project team is aligned right from the start.

If you’re not using a form or if you’re in an ad-hoc meeting, these aspects can be discussed openly. Either way, this ‘Goals & Expectations’ step should be the launching pad for the rest of the project, giving everyone a clear direction and a shared understanding of what success looks like.

Example questions:

• What is the specific outcome you’re looking to achieve with this data analysis?
• How does this goal align with departmental or organizational objectives?
• Are there any benchmarks, KPIs, or sources of truth that this analysis should align with?
• What is your timeline for achieving this goal (or a flat-out due date)?
• Question for internal team: What assumptions are we making about the data or a process that need validation?

Remember, you’re not just gathering information here—you’re creating a roadmap for every decision and action that will follow. The next logical step is to look at the data itself. After all, high-quality, ethical, and accurate data is the backbone of any successful analytics project.

Laying the Data Groundwork

Moving onto the data itself, this step is critical to ensure that the foundation of our project is rock-solid. Without reliable data, even the best-laid plans can crumble. There are several key considerations here, broken down into data sources, quality, and ethics.

Data Sources

Identifying the right data sources early on will save you from potential roadblocks later. Is the data you need readily available, or will it require additional effort to obtain? This can be a good time to familiarize yourself with the data if you’re not already. Getting a sense of the variables and how they relate can be handy as you start thinking through any request.

• Question for stakeholders: Where will the data come from? Known limitations or issues with it?
• Question for internal team: Do we have access to these data sources, and are they reliable?

Data Quality

Not all data is created equal, and poor-quality data can drastically affect your results. You’ll want to evaluate the accuracy, consistency, and completeness of the data. A little exploratory data analysis done in stages can be helpful.

• Question for stakeholders: How current does the data need to be? How accurate does it need to be?
  • I don’t like that accurate questions, but I recognize that sometimes quick & dirty is valued over a high degree of certainty that would take more time.
• Question for internal team: Do we have the tools and skills to clean and prep the data to fit the objectives? Is the granularity the project needs, available?

Ethical Considerations

Data ethics should never be an afterthought. From privacy concerns to data bias, ethical considerations should be front and center.

• Question for stakeholders: Are there any sensitive or restricted data involved in this analysis?
• Question for internal team: What safeguards can we put in place to ensure ethical handling of data?

This step may necessitate a meeting or even multiple discussions to make sure everyone is on the same page regarding the data that will be the backbone of your analysis. Whether you’re capturing this information through a form or through meetings, making sure that you’re clear about these data considerations is essential for the project’s success.

Deciding on Analytic Strategy: Methods, Tools, and Validation

The next logical step after establishing a strong data foundation is to agree on how the data will be analyzed. This is where your team’s expertise really comes into play. It’s also where you’ll decide on the specific analytical methods that will drive you toward your project’s objectives.

Analytical Methods

Are you going for a descriptive analysis that tells you ‘what’ is happening, or are you aiming for a predictive model that will inform ‘what could happen?’ The choice of analytical method should align with the project goals you’ve already outlined, and while I am sharing suggestions that you can ask stakeholders, you may not want to. You understand your customers and organization better than I do!

• Question for stakeholders: What type of analysis do you envision? Descriptive, predictive, prescriptive, or something else?
• Question for internal team: Do we have the capability to execute this type of analysis effectively?

Choosing the Right Tools

In today’s data landscape, the choice of analytical tools can be overwhelming. From Python to R to specialized software, the options are numerous. The best tool for the job should fit both the analytical method and the skillset of the team. Plus, how are the results expected back? Are we talking about a PowerPoint presentation, a quick spreadsheet or an ongoing Tableau dashboard?

• Question for stakeholders: Are there any tools or platforms you prefer for this analysis? Any you’d like to avoid?
• Question for internal team: What tools do we have available & experience with that will accommodate the request?

Validation and Verification

Before diving deep into analysis, it’s crucial to determine how you’ll validate the results. This is the ‘reality check’ that ensures your analysis meets the project’s goals and expectations, and that it’s ultimately accurate.

• Question for stakeholders: How will you measure the success of this analysis?
• Question for internal team: What validation techniques can we apply to ensure the results are robust?

Remember, the methods and tools you choose will guide the analytical work. It’s not just about crunching numbers; it’s about deriving meaningful insights that will achieve the project’s goals. This section may also warrant a meeting or further discussions to make sure all parties are clear on the analytical direction of the project.

Now, you’re not just ready to start your data project; you’re prepared to execute it with precision and purpose.

Common Mistakes the Right Questions Can Fix

Ignoring the initial steps and plowing right into the data project can seem tempting, especially when deadlines loom and pressure is high. However, skipping the preliminary questions can lead to a host of problems. Let’s delve into some of the most frequent mistakes:

Overlooking Stakeholder Involvement

The Error: Not involving stakeholders from the get-go can lead to misguided project goals and eventual dissatisfaction.

The Fix: Early engagement ensures that all parties have their expectations aligned and are invested in the project’s outcome.

Undervaluing Data Quality

The Error: Assuming all data is good data can severely impact your analysis and conclusions.

The Fix: Take the time to scrutinize your data sources for reliability, accuracy, and completeness. Exploratory Data Analysis (EDA) is often overlooked or minimally done in favor of speed. Speed vs accuracy is a real trade-off that can impact the perceived success of your output – whether it’s a one-off or a dashboard.

Vague Goals and Expectations

The Error: Aiming for an ill-defined outcome like “improve customer experience” can lead to a drifting, aimless project.

The Fix: Set SMART goals that align with both stakeholder expectations and organizational KPIs.

Overlooking Ethical Considerations

The Error: Ignoring ethics until it becomes a problem can derail your project and harm your organization’s reputation.

The Fix: Address ethical considerations upfront and establish protocols for responsible data handling.

Why Preliminary Questions Matter

You’ve set your goals, assessed your data foundation, and decided on your analytical strategy. These questions to ask for data analysis aren’t mere checkboxes; they’re pillars upon which your entire data project will stand. By asking the right questions upfront, you’re doing more than just planning—you’re actively building a resilient framework that can adapt to challenges and new information.

Asking these initial questions may seem like a lot of work, but it’s work that pays off exponentially. You’ll have a well-defined roadmap that aligns with both stakeholders and internal teams, ensuring a smoother project execution. And most importantly, you’ll have the assurance that comes from knowing you’re not just ready to start your data project—you’re prepared to make it a success.

Whether you’re a data veteran or embarking on your first data project, never underestimate the power of asking the right questions at the start. Your future self, your team, and your stakeholders will thank you.