Tagged: analytics
From North Star to Unit Economics
This is a new series that focuses on my thoughts and systems around value stream management. I call it ValueOps, a system I have been building long before AgenticOps.
AI has given me a gift by allowing me to expand my exploration of topics I have been exploring for years. Some may call this post AI slop because I allow AI to cook, but I am most certainly still the chef. The recipes are mine, the ingredients (my thoughts) are mine, and all words pass my taste and quality tests.
I hope you and your agent gets something out of this and it helps you to continuously improve the systems you care about.
Bon Appetite
I have always liked North Star Metrics. They force a team to stop measuring everything and decide what one thing actually points toward value.
That is useful. It is also not enough.
I have seen teams rally around a metric that felt right, moved up, and still did not explain whether the business was getting healthier. The chart improved. The economics did not. Or worse, nobody could tell if the economics improved because the North Star lived in one system and the money lived somewhere else.
So the problem is not North Star thinking. The problem is where it usually stops.
A North Star gives direction. Unit economics gives economic truth. The gap between them is the operational event that can be measured, followed, converted, costed, retained, and tested.
That event is what I call a Value Fact.
What the North Star Got Right
Amplitude describes the North Star Metric as the metric that best captures the value customers get from a product. That is the right instinct. The metric should not be vanity activity. It should be close to the thing the customer values.
Reforge pushes the idea further by breaking North Star Metrics into unit of value, quality, and frequency. That matters because not every action counts. A signup is not the same as an activated user. A visit is not the same as a reviewer. A trial is not the same as a trial with real usage.
That is the part I want to keep.
A good North Star is not revenue. Revenue is late. It tells you what already happened. A good North Star is upstream from revenue, but not so far upstream that it becomes noise.
It is the best observable proxy for value creation before the financial result fully arrives.
Airbnb’s common example is nights booked. That works because a night booked is not just activity. It means the marketplace connected demand and supply. A guest found a place to stay. A host received a booking. The business created the conditions for revenue.
That is the shape.
Where the North Star Falls Short
Here is where I think the usual North Star conversation gets weak.
It helps the team align, but it does not always make the metric accountable.
You can pick a North Star, put it on a dashboard, and still not know what one more unit is worth. You may not know what it cost to create. You may not know whether it converts. You may not know whether the customers it produces stay long enough to justify the investment.
So the team gets direction, but the operator still lacks economics.
Now, that might sound unfair. A North Star is not supposed to be a full financial model. Fair. I agree.
But if the metric is going to guide product strategy, resource allocation, experiments, and operating decisions, then at some point it has to cross the bridge into economics.
Otherwise the organization is optimizing a belief.
Maybe it is a good belief. Maybe not.
The Bridge Is the Value Fact
ValueOps starts with a narrower claim.
Do not start with all the metrics. Do not start with revenue. Do not start with a dashboard.
Start with the countable operational fact that represents unrealized value.
For a SaaS company, that might be an activated trial workspace. For a marketplace, it might be a completed match. For a services firm, it might be a completed job that creates a follow-on opportunity. For a support product, it might be a real ticket resolved by the agent.
The name matters because it separates two ideas that usually blur together.
North Star is the strategic role.
Value Fact is the measurement object.
The North Star tells the team, this is the event we believe points toward value. The Value Fact model says, prove it. Count it. Cohort it. Attach dimensions. Track conversion. Measure the value. Measure the cost. Then see whether the belief survives contact with the business.
Here is the path.

That is the move from product strategy to unit economics.
Not because the North Star was wrong. Because it was unfinished.
What Has to Be True
For a North Star to become a Value Fact, it has to pass a few tests.
It has to be countable. The event either happened or it did not.
It has to be cohortable. You can follow the facts from a period forward and see what happened to them.
It has to be attributable. You know which channel, segment, workflow, team, region, or operating path produced it.
It has to be convertible. There is a later event where the fact becomes delivered value.
It has to be costable. You can measure what it took to produce and convert the fact.
It has to be economically meaningful. One more fact should imply some future value, even if that value has not arrived yet.
If the metric cannot pass those tests, it may still be useful. But I would be careful calling it the operating center of the business.
The Series
This series builds the measurement system from that point.
Value Impact asks whether the operation is getting better at turning facts into value.
Value Efficiency asks what it costs to produce and convert those facts.
Value Ratio puts value and cost in direct relationship.
Value Payback asks how long it takes to recover the investment.
Value Retention asks whether the value stays long enough to matter.
Value Margin asks how much of the revenue the business actually keeps.
Then the final post reduces the system to seven levers and an engine. Fact Volume, Cost Per Fact, Conversion Rate, Cost of Conversion, Revenue Rate, Retention Rate, Cost to Serve.
That is the control surface.
The math is not the hard part. The hard part is picking the right fact and refusing to let it remain a slogan.
The Claim
ValueOps does not replace North Star thinking.
It makes North Star thinking accountable.
The North Star gives the product direction. The Value Fact gives the measurement system something to operate on. The models turn that fact into economics.
That is the bridge I have been looking for.
From North Star to Unit Economics.
Let’s talk about it.
Every Product Needs a North Star Metric
How to Choose and Measure North Star Metrics
Next: Value Impact
Optimizing the Software Delivery Pipeline: Deployment Metrics
Currently, I have no way of easily determining what build version is deployed to an environment. This made me take more interest in metrics about deployments, we basically have none. I can look at the the CD (continuous deployment) server and see what time a deployment was done and I can look at the builds on the server and sort of deduce which build was deployed, but I have to manually check the server to verify my assumptions. I wondered what else I am missing. Am I flying blind, should I know more?
Metrics in the Software Delivery Pipeline
I am part of a work group that is exploring software quality metrics. So, my first instinct was to think about deployment quality metrics. After some soul searching, I decided what would be most helpful to me is to know where our bottle necks are. We have an assembly line or pipeline that consists of various stages our software goes through as it makes its way to public consumption. Develop, build, deploy, test, and release are the major phases of our software delivery pipeline (I am not including planning or analysis right now as that is another animal).
I believe that metrics that focus on reducing time in our software delivery pipeline will be more effective than just focusing on reducing defects or increasing quality. If we can reduce defects or increase quality in faster delivery iterations, the effect of defects and poor quality will have less of an impact. This is the point of quality metrics in the first place, reducing the effects of poor quality on our customers and the business. Focusing on reducing time in the pipeline also supports our quality initiatives as the tools to reduce time, like automated CI and testing, not only reduce iteration time, but improve quality. Faster release iterations will allow us to address quality issues quicker. This is not to say that other metrics should be ignored. I just think that since we have no real metrics at the moment starting with metrics that support speeding up the pipe is a worthy first step.
Deployment Metrics
Back to the point. What metrics should I capture for deployments. If my goal is to increase throughput in the pipeline, I need to identify bottlenecks. So, I need some timing data.
- How long does deployment take?
- How long do the individual deployment steps take?
- How do we report this over time so we can identify issues?
This is pretty simple and I can extract it from the deployment log on the server. Reporting would be just a matter of querying this data and displaying deployment time totals over time.
Additional Deployment Metrics
In addition to the timing data it may be worthwhile to capture additional metrics like the size of deployment. Deploying involves pushing packages across wires and the size of the packages can have an effect on deployment time. Issues with individual servers can affect deployment time so, knowing the servers being deployed to can help identify server issues. With the timing data, we can also capture
- The version of the build being deployed
- The environment being deployed to
- The individual servers being deployed to
- The size and version of the packages being deployed to a server
Deployment Data
So, my first iteration of metrics center around timing, but would also have other data to give a more robust picture of deployments. This is a naive first draft of what the data schema could look like. I would suspect that this can all be captured on most CI/CD servers and augmented with data generated by the reporting tool:
- Deployment Id – a unique identifier for the deployment, generated by the reporting tool
- Environment Id – a unique identifier for the environment deployed to, generated by the reporting tool
- Build Version – build version should be the version captured on the server
- Timestamp – timestamp is the date/time the deployment record was created
- Start – the date/time the deployment started
- End – the date/time the deployment completed
- Tasks – tasks are the individual steps taken by the deployment script; it is possible that there is only one step, it all depends on how deployment is scripted
- Deployment Task Id – a unique identifier for the task, generated by the reporting tool
- Server Id – a unique identifier for the physical server deployed to, generated by the reporting tool
- Packages – packages represent the group of files pushed to the server, this is normally a zip or NuGet package in my scenarios
- Package Version – the version of the package being pushed, this may be different than the software version and is generated outside of the reporting tool
- Package Size – the physical size of the package in KB or MB (not sure which is better)
- Start – the date/time the deployment to the server started
- End – the date/time the deployment to the server ended
Imagine the above as some beautiful XML, JSON, or ProtoBuf, because I am too lazy to write it.
If my goal is to increase throughput in the pipe I should probably think about a higher level of abstraction in the hierarchy so that I can relate metrics from other parts of the pipeline. For now I will focus on this as a first step to prove that this is doable and provides some value.
All I need to do is a create data parsing tool that can be called by the deployment server once a deployment is done. The tool will receive the server log and store it, parse the log and generate a data structure similar to above, then store the data in a database. Then I have to create a reporting tool that can present graphs and charts of the data for easy analysis. Lastly, create an API that will allow other tools to consume the data. This maybe a job for CQRS and event sourcing. Easy right :). I know there is a tool for that, but I am a sucker for punishment.
Conclusion
This post will take more time than I thought so I will make this a series. I will cover my thoughts on metrics for development, build, test, and release in upcoming posts (if I can remember). Then possibly some posts on my thoughts on how the metrics and tools can be used to optimize the pipeline. Pretty ambitious, but sounds like fun to me.