Haus

Our Head of Science, Joe Wyer, shares a powerful lesson from his time at Amazon: In the early days of advanced marketing analytics, there was significant debate within Amazon about the value and implementation of incrementality testing and causal inference. These methods were new, complex, and sometimes yielded results that conflicted with traditional metrics. As disagreements escalated, the issue eventually reached Jeff Bezos himself. When faced with this debate about incrementality and causal inference, Bezos' response was clear and forward-thinking: "The companies that do this will win. The companies that don't will lose." This wasn't just talk. Amazon committed to these methods, and it's paid off. Here's the thing: for years, only giants like Amazon and Netflix had the resources to leverage these advanced analytics tools. That created a massive competitive advantage. At Haus, we believe this shouldn't be the case. We're on a mission to democratize these powerful tools, making them accessible to businesses of all sizes. Why? Because as Bezos predicted, embracing data-driven decision-making isn't just an advantage anymore - it's becoming essential for survival in today's market. Our journey hasn't been easy. We've faced countless challenges, uncovered weird corner cases, and had our fair share of head-scratching moments. But that's exactly why our tools are so robust. We don't just "peanut butter" results and call it a day. We dig deep, we solve problems, and we build solutions that work in the real world, not just in theory.

When attribution and incrementality disagree, what's a marketer to do? This scenario is more common than you might think, and it can lead to some tricky decision-making. Here's a real-world situation our Head of Science, Joe Wyer, has encountered in the industry: Imagine a large brand running an incrementality test on their brand search. The results show almost no impact, and upon retesting, they again find zero incrementality. Now, the brand is working with an agency that's evaluated based on attribution-based cost-per-acquisition (CPA) targets. Brand search is the only channel consistently meeting these CPA goals, allowing the agency to continue running other channels. This creates a problem: tests show brand search isn't adding new value, but reports say it's crucial for hitting targets. It's a clash between new insights and old ways of measuring success. This conflict puts marketers in a tough spot when deciding where to invest their budget. How do we navigate this? 1. Align on decision frameworks before seeing results 2. Be willing to challenge assumptions and rethink established practices 3. Use attribution as a starting point, not the final word 4. Prioritize incrementality testing on your biggest bets 5. Understand the relationship between attribution and incrementality for your specific business 6. Continuously test and adapt as platforms and customer behaviors change The key takeaway is that you shouldn’t let attribution metrics alone drive your strategy. By embracing incrementality testing and being open to challenging results, you can uncover new opportunities and optimize your marketing mix for true business impact. One brand we worked with did just that. They shifted away from non-incremental spend and discovered new channels (like YouTube) that drove significant incremental results.

Haus' Head of Strategy Olivia Kory sums it up well. Link in comments to the Haus Viewpoint – as relevant today as ever before.

We all might be too obsessed with statistical significance… Here's a scenario you might recognize: You run a test. The results show an impressive return on ad spend (ROAS) of 3. But… the p-value isn't below 0.05. Suddenly, you're told you can't reject the null hypothesis. What does this mean? According to traditional statistics, you can't say with certainty that your campaign had any effect at all. Here's the thing: this approach often leads to wheel-spinning and missed opportunities. At Haus, we approach this differently. Our Head of Science, Joe Wyer, Ph.D., brings a unique outlook to this problem. He draws a parallel to fields like professional poker and investment: "In these domains, decisions are made based on expected value and ROI, not just statistical significance," Joe explains. "This approach to decision-making under uncertainty can offer valuable insights for marketing measurement." This comparison highlights an important point: in fast-moving business environments, waiting for perfect statistical certainty can sometimes be counterproductive. Instead, we focus on: 1. Calculating expected value 2. Considering the range of potential returns 3. Making informed decisions based on the best available data This approach allows for more agile decision-making without sacrificing analytical rigor. It's about finding the right balance between data-driven insights and practical business needs. In the fast-paced marketing world of today, a calculated risk based on solid data can be better than perfect certainty that comes too late.

When it comes to measuring marketing impact, are you looking at the right KPIs? If you’re not sure, you’re not alone. Many businesses struggle with this, and it’s costing them valuable insights. Let’s break it down with some real-world scenarios: 1. Subscription businesses: Are you measuring paid conversions too soon after a free trial starts? If so, you might be missing the full impact of your acquisition campaigns. 2. High-priced products: Is your analysis window too short to capture the entire customer journey? For big-ticket items, customers often take longer to decide, so a short analysis period might miss delayed conversions, underestimating your marketing impact. 3. Multi-product businesses: Are you looking at top-line revenue when you should be segmenting by product category? This granular view helps isolate the impact of specific marketing efforts on different product lines. At Haus, we tackle these challenges with strategies like: - Lagged KPIs - Post-treatment windows - Short-term engagement signals - Product category segmentation These approaches make sure you're capturing the true impact of your marketing efforts, including potential halo effects.

What are the best practices for setting up and analyzing mobile app marketing experiments? What unique challenges do marketers experience when measuring effectiveness for mobile apps compared to other platforms? How can you attribute conversions and in-app events to specific marketing channels? And what KPIs should you be tracking? These are some of the questions Olivia Kory and Zach Epstein will be asking Noa Gutterman, Senior Director of Growth at TextNow and former director of marketing at VSCO, at our next Open Haus on July 15. We’ll also talk about: - The role of PMAX and UAC in paid user acquisition - How to leverage incrementality testing to optimize mobile app marketing spend - Techniques for measuring and improving user retention and lifetime value - Best practices for setting up and analyzing mobile app marketing experiments Join us for a 45' live Zoom event with Q&A at the end for questions around testing, experiments, incrementality, etc. Link to register down in the comments!

With CACs on the rise, allocating large budgets to customer acquisition for your mobile app just to have a “leaky bucket” with great churn is a losing strategy. Now more than ever, it’s important for marketers to understand how to measure and improve user retention and lifetime value. That’s why we’re so excited to bring in Noa Gutterman, Senior Director of Growth at TextNow and former director of marketing at VSCO, to our July 15 Open Haus with Zach Epstein and Olivia Kory to chat about what effective measurement for mobile apps looks like. We’ll also talk about: - Unique challenges in measuring mobile app marketing effectiveness and how to overcome them - The role of PMAX and UAC in paid user acquisition - Key metrics and KPIs for evaluating user acquisition campaigns - How to leverage incrementality testing to optimize mobile app marketing spend - Best practices for setting up and analyzing mobile app marketing experiments Join us for a 45' live Zoom event with a Q&A at the end for questions around testing, experiments, incrementality, etc. Link to register down in the comments:

Noa Gutterman, Senior Director of Growth at TextNow and former director of marketing at VSCO will join Zach Epstein and Olivia Kory for our July 15th Open Haus to discuss how to master mobile app measurement: unique challenges marketers are facing, attributing conversions to marketing channels, KPIs, paid user acquisition, and how to set up useful mobile app marketing experiments. Join us for the 45' live Zoom event with an added Q&A at the end for questions around testing, experiments, incrementality, etc. See you there! Registration here 👉 https://lnkd.in/gHJirNQx

700%: That’s how much social and video marketing spend outshine search when it comes to influencing sales on Amazon compared to DTC, according to internal Haus data. Take it with a grain of salt – every brand is different, and it’s key to test for your unique business. With Amazon Prime Day 2024 right around the corner, there’s no better time to dig into what our trove of customer experiment data reveals. Here’s a preview: - Across media channels and ad formats, 97% of incrementality tests in our database show a non-zero, positive lift in Amazon sales. - One out of every six tests show greater sales lift on Amazon than on DTC. - 83% of experiments drive a >10% halo effect on Amazon sales. Dive into the full insights: ⤵️

In marketing analytics, there's often pressure to present data in a way that aligns with stakeholders' expectations. However, this can lead to challenges in making truly data-driven decisions. At Haus, we've developed an approach to address this common issue: hands-free analysis. What does this mean? - Our pipelines and analysis configurations are set up to run without human intervention at the end. - No knob-turning or lever-pushing to manipulate outcomes. - Clear decision gates: "If the data looks like X, we do Y." This approach ensures: - Consistency across all analyses. - Elimination of bias from result-seeking behavior. - Trustworthy insights that truly inform business decisions. While this might be common in smaller businesses, larger companies often struggle with the temptation to "massage" the data. Our hands-free method removes that possibility entirely. The bottom line is that marketing experiments aren't conducted to tell you what you want to hear – they’re meant to reveal the hard truths that can help you optimize your marketing budget.