Testing In The Wind Tunnel with Stryd’s New Running Power Meter

DSC_1894

It’s been now 4 years since Stryd first landed on the market as a running power meter. Back then, it was initially a chest strap, though the company shifted to a single footpod design about three years ago. Now, with their next generation footpod they’ve incorporating wind detection hardware and algorithms to determine the impact of wind on a runner.

Now before we get too far into this, some will try and argue that this isn’t a ‘power meter’. However, that’s largely a backwards and dated way of thinking. Ultimately one has to realize that every power meter is part hardware and part software algorithms. Whether or not this one uses wind and accelerometers to measure power versus some element of direct force is inconsequential. What matters is whether or not it can do it accurately. The cycling power meter and trainer market is littered with examples of crappy so-called ‘direct force’ power meters, and very functional non-direct force power meters. Some arbitrary distinction no longer mattes in accuracy. Either it sucks or it doesn’t. And no, the argument of “it’s at least consistent” doesn’t fly in these parts.

However, don’t mistake the preceding paragraph as me saying Stryd is accurate. Oh god no. Not because I think they’re inaccurate, nor because I think they are accurate. But rather, because I haven’t decided yet. Mind you, I suspect there’s nobody in this world that has tried more running power meters than I’ve donned, the vast majority of them from companies that have either never made it to market, or not survived their first year. So the fact that Stryd is still around is saying something. But I also wouldn’t call them the golden standard either, since I don’t believe such a term exists for running power meters yet for a variety of reasons (or many other devices for that matter).

Got all that caveating? Good. Let’s talk tech then. Don’t worry, we’ll get back to accuracy in a bit.

What’s New:

DSC_1904

In many ways you’ll look at the exterior of this pod and assume it’s the same (new one is left with orange clip, right with black clip is old). After all, it sure looks like it. But internally there’s some substantial hardware changes that make the existing pods unable to include this functionality via simple firmware update. So no, the existing Stryd units don’t have the correct hardware (namely the temperature and humidity sensors, as well as clip/shell) to enable this via simple firmware update.

Here’s what’s changed though, according to Stryd (and they walked me through the internals of an actual pod as well earlier this spring):

– New shoe clip designed to properly funnel wind to sensor
– Added temperature and humidity sensors
– Added magnetometer for improved accuracy and future features
– Increased storage 64x over previous unit, to aide in higher storage requirements for captured data, and allowing more data to be captured
– Brighter & more clear LED status light
– Raised the price from $199 to $219 (but now it includes shipping, so basically a wash)
– Ditched wireless charging in favor of small charging cradle
– Increased antenna range to deal with Garmin’s poor Fenix 5 lineup antenna reception

Ok, the last one they didn’t officially specify Garmin’s Fenix 5, officially it says “no matter your phone or sports watch”, but everyone knows that’s really what they mean. Unfortunately, it’s probably a bit late for that for most people.

In any case, the biggest ticket item here people care about is that the unit itself now accounts for the impacts of wind. Up until now, it was the single biggest issue with Stryd (and most other running power meters): It didn’t account for headwinds or tailwinds. And no matter how many times these companies tried to pretend that headwinds didn’t matter to running power, they were seriously talking to the wrong person. As a poor sap that has run an entire one-way marathon into a non-stop headwind once, I knew full well that a headwind made a huge difference on race day. Never understood pretending it didn’t.

POD explod

 

stryd-captures-the-wind

The new Stryd pod has changed the positioning of the clip to allow for better airflow into the pod, which in turn allows them to measure wind more accurately. That meant not just changing the clip and case, but also then dealing with better waterproofing to the pod, including a new membrane inside the pod to ensure that changes in pressure get in, but not water running through puddles. Starting first, here’s the pod itself, compared to the old pod. This is looking at it upside-down with the front facing the bottom:

DSC_1905

Next, you can see most notably the new hole/gap in the front of the orange pod clip, which allows the airflow to come into the pod:

DSC_1911

Note that the back of the clip was also changed, such that it forces you to always clip it in back first, whereas the existing pod could technically be done both ways but was apparently supposed to only be done one way (and the other could lead to damage).

DSC_1785

For the charging aspect, gone is the oversized wireless charging mat, and instead we’ve got a small cradle. This is a much better solution, which in turn connects to micro-USB. Would have liked to seen USB-C here, but at least it’s not the near shoe-sized charging pad anymore that you had to bring when you traveled. At right below, you can see the contact ports for the charging dock next to the pod.

DSC_1778

 

DSC_1775

Beyond that, all interface aspects are the same. It still connects to your Garmin, Suunto, Polar, or Apple Watch wearables through their respective apps or native integrations. We can debate the pros and cons of those in the full review (it’s not as easy as saying one is absolutely better than the others, as there’s lots of nuances to each). And the data still shows up the same way as well, showing you wattage on your watch depending on the exact device/format/app/etc…

DSC_1758

Again, all things we can discuss in more depth down the road, but all things that haven’t changed. Note that as of this very second, whether or not it’ll display wind speed is a bit unclear. They have a prototype app that displays/records it, but it’s not in the production app yet. They do understand the value there in that though, so hopefully that makes the cut sooner rather than later.

Testing in The Tunnel:

DSC_1919

I’ll be honest: I mostly dread going to the wind tunnel.

Sure, it looks cool and all, but it’s fraught with failure. Last fall I spent time with some of the smartest people in the sports aerodynamics industry in a wind tunnel. And the net output of that was basically a whiteboard full of ways one could fail doing tests in a wind tunnel. Not ways the product could fail, but just ways the process of testing could fail.

So while I enjoy the concept of wind tunnel days, I also despise them for the reality of a damned if you do, damned if you don’t. I’ve got not doubt there will be someone in the comments who said we’ve done something wrong. And that’s likely, in fact, we found a few things that didn’t work setup-wise. Some of them obvious, some of them less so.

Ultimately we put about 30 kilometers of running today on a treadmill in the tunnel, and oh, we had some ‘smart people’ there. From the aerodynamics side of the house, the full time engineers at the DNW Wind Tunnels (Germany-Dutch Wind Tunnels) helped with logistics and running the tests, as well as giving thoughts on setup/etc. Their job is to manage aerodynamics tests almost every day of the year for everything from special classified military projects, to commercial airlines, to silly running things.

DSC_1933

Next, there was Ron Van Megen & Hans Van Dijk, behind the book The Secret of Running: Maximum Performance Gains Through Effective Power Metering and Training Analysis (Meyer & Meyer Premium). They had a bunch of data from their research and others research around the area of running efficiency, including numerous white papers on running with power meters. Certainly Stryd brought them there, but I didn’t see any bias towards Stryd in our discussions.

DSC_1936

They were pretty even in my discussions with them. We discussed some of the variances between running power meters and more importantly their algorithms. They had some good insights, specifically into the never-ending debate on whether one unit is right or not. Most of which simply boils down to different units actually reporting slightly different metrics all bundled under the heading ‘running power’. We talked specifically for example about how Stryd relates to Polar & Garmin running power algorithms. There was no discussion of one being the absolute right answer, rather, just different ways of presenting similar data. You’ll find numerous white papers they’ve written on the topic here. But the super short oversimplified version is that Garmin and Polar include the elastic recoil/rebound output in their power numbers, whereas Stryd doesn’t. Some pros and cons to both methods.

DSC_1938

And finally, Stryd themselves was there in the tunnel as well as connected via a group Skype call back to their headquarters in Boulder, where they were enjoying being up at a god-awful hour of the morning.

DSC_1762

Oh, right, and there was me too. I basically just ran, mostly with the goal of not falling off the back of the treadmill with near-tropical storm winds and ending up in a turbine somewhere.

DSC_1754

The test protocol was simple. First, it involved relocating my treadmill from the DCR Cave to the secured wind tunnel facility/base over an hour away. I gave them a key to the studio, went on vacation, and said ‘Y’all have fun with that.’ Having moved said treadmill some four times now, I’ve long since learned I’d rather shovel cow pies than move that twirling finger-slicing beast around. I noticed the Stryd person was wearing a wrist injury brace after they moved it. No idea if those two were connected.

But they got it moved, and then got it bolted into the wind tunnel. While this one was not the supersonic tunnel (which is capable of speeds up to Mach 1.3), this particular tunnel could still unleash precise winds up to 288KPH (179MPH). Homey didn’t want my treadmill becoming hurricane fodder.

2019-06-27 13.20.21

 

2019-06-27 13.23.36

Next, the scientist guys proposed a specific protocol. It was straight forward and pretty easy to validate the basics of the Stryd device from a wind perspective.

1. Run at 10KPH/6.2MPH on the treadmill (the pace specifically a bit slow/easy to allow us to account for higher winds later in the test)
2. Start at 0 m/s wind speed. Run for ~3 minutes
3. Increase wind speed in steps of 3 m/s, run for 3 minutes
4. Max wind speed is 15 m/s (54KPH/33.6KPH)
5. Repeat for 3 runners (at least).
6. If time allowed, repeat test at a 45* yaw angle (cross-wind basically)

And basically, that’s exactly what we did. In addition to running with the new Stryd pod, I also ran concurrently with my older Stryd pod (the ‘current’ generation prior to today). Both were configured with exactly the same weight, with one on each shoe. First however, a couple of ‘known’ issues/notables/etc… with our plan:

1. We didn’t know exactly how the airflow over the lip or posts of the treadmill would impact actual wind speed that hit the shoes
2. We knew that stabilization of the tunnel wind speeds at this specific velocity would take about 30-45-seconds, so we budgeted 60 seconds for safety
3. We ended up going with 10.7KPH on the treadmill as the displayed speed, as that’s what the Stryd said was 10KPH, just to keep later math easier.
4. We didn’t know what we didn’t know.

I say the last one, because frankly there’s very few people in this entire world that have put treadmills inside wind tunnels. And even fewer that have bothered to measure it in a useful manner. And yet fewer still that have shared that data and their learnings. Sometimes ya gotta try in order to learn.

Lastly, I collected *all my own data*. By that I mean that Stryd had no control/access to my data before I did. I paired their new sensor directly to my watches and downloaded the data directly from my Garmin watches (MARQ Athlete and FR935) to my computer (.FIT files). I also used one of their watches as a backup (FR245) with their latest app. I note that because no post-processing was occurring here. It was all real-time measurements. Said differently: They couldn’t fudge the numbers before I could see them.

Without further rambling, here’s the only chart you care about. It shows the non-wind capable existing Stryd footpod power meter against the wind-capable unit, from a power output standpoint. You’ll note that the older Stryd unit basically stays ‘flat’ the entire time (as the treadmill pace was constant), whereas the newer unit rises as the wind speed rises.

image

Super cool, huh? So, to roughly simplify this, here’s some key points:

At 0 KPH/0MPH: Both units were within a few watts
At 10.8 KPH/6.7 MPH:  Very little difference as well, only a few watts higher wind resistance
At 32.4 KPH/20.1 MPH: Big jump here, roughly a 50w increase in power required, and about a 10bpm HR rise
At 43.2KPH/26.8 MPH: Another big jump, roughly a 90-100w increase over baseline, and a 20bpm HR rise over baseline

I’ve skipped a few lines in the summary above, though you can see them in the chart above, or also at the full dataset here. We should have the wind tunnel data facility by tomorrow, which shows the super-precise measured tunnel speeds, though the approx speeds are rounded/listed above.

So next, what about wind speed? For that Stryd had a special Garmin Connect IQ app that showed wind speed, both real-time and a slightly corrected one based on some of their algorithms. Note the wind tunnel speed was measured by more pitot tubes than I could count. Seriously, I stopped counting after getting to something like 8-10 pitot tubes around the tunnel. Here’s how those numbers played out (I’m using approximate, because there’s slight variations in the numbers depending on which points in the 3-minute steps we use). Also, I’m using MPH because it’s just the mode the app was in, since I’m tired of re-converting numbers three ways right now and for this bit you’re just looking at variance anyway.

Wind Tunnel Speed (MPH) vs Stryd Reported Wind Speed (MPH):

0 MPH | 0 MPH
6.7MPH | ~6.8MPH
13.4MPH | ~11.9MPH
20.1MPH | ~20.3MPH
26.8MPH | ~29.3MPH
33.6MPH | ~~40MPH

As you can see, at some wind speeds it was pretty close, and others less so. So why the variation? Well, likely the test setup is one factor. In an ideal setup the treadmill itself would be flush with the floor.  In this case, the wind is going to create a zone both above and below the treadmill that can cause funkiness in airflow speeds right about the part where my foot is. In fact, all three of us running notices that if we shifted slightly forward or back on the treadmill belt, it’d change the wind speeds slightly.

image

Still, for the majority of speeds we cared about (lower speeds at 20MPH or so), it was very very very close.

Next, what about cross-winds? Such as an offset wind at 45* to the runner? No problem, the wind tunnel systems can rotate the treadmill automagically:

DSC_1928

Except, that immediately introduced a new issue. The supports of the display/control portion of the treadmill created a blocker for the footpod, but not in the way you’d probably thing. However it was also something that the resident wind tunnel engineers expected would happen. The output wattage by Stryd was far higher than it should have been, because the measured speed was as well. Basically, the air flowing around the pole was going faster to account for the pole. We can get into all sorts of aerodynamics on why this happens, but there’s an entire internet out there for you if you want to.

The net result was that the data we collected during this test was throwaway due to the test setup. Stryd did show me some data they collected in another wind tunnel with two other athletes where they were able to remove the support pillars of the treadmill, and in that case it showed the power output at approx 70% of the non-yaw headwind power (which is where it should be math-wise). I don’t really have a reason to doubt them, but inversely I have no method to prove it out here myself with the gear I have.

Finally, before folks ask – while I have actually run briefly with a test unit outside in the real world back earlier this spring, it was an early prototype and didn’t record data (I could only see it real-time). Further, I didn’t leave this test with a pod of my own. So I can’t do any more tests until mid-July when they expect to have a final production pod ready. The good news is that winds are plentiful here in the Netherlands, so it’s easy enough to test once I get one. Though by the same token, the data won’t likely be as super clean as the above.

Also, while I had hoped we’d be able to test flipping the treadmill around and doing a tailwind, we simply ran out of time for that (the tunnel operates on a super strict time table). Fear not, I’ll be doing tailwind tests once I get a pod outdoors. After all, those are the best kind of tests.

Going Forward:

DSC_1914

So, where do we stand?

Well, at first glance after some initial testing I’d say that the results are promising. Lack of accounting for wind has always been one of my biggest sticking points with running power (alongside lack of consistency between vendors). This may indeed solve for that. While tunnels are great for proving out specific use test scenarios such as this, I still prefer significant on-road time outdoors with variable conditions. Including tailwinds and more cross-wind work. Still, the numbers we see here match both the wind power-science math numbers as well as most of the practical wind speeds from the tunnel itself.

These new Stryd pods will start shipping late July according to the company, which I suspect is probably in the ballpark. Though my gut feeling is probably closer to early August before folks see actually them. Just a hunch. Either way, Stryd’s hoping to have a final production review unit to me by mid-July to start digging into more tests with. Which would then put a review sometime in late August, maybe mid-August. Really just depends on when a unit arrives.

With that – thanks for reading~