Live Blogging Developer Week, 2020, Day 2
Why the Dev Tools Market is Designed to Confuse You, 4:00 Pacific
This is a talk by Chris Riley, of Splunk. And it starts off with a great hook, which is “dev tools market is really confusing.”
It’s so confusing, in fact, that here’s a screenshot of the tools available for you to choose from:
Technical founders are good at building tools to solve a problem, but not as good at forming a goto market strategy. The result is a never-ending proliferation of tools.
This is exacerbated by the fact that we, as techies, love to tinker. As Chris suggests in his slide, for a lot of dev teams, tools are “junk food.” We should perhaps get back to dev tools as veggies.
Which Tool is Best?
How do you know if a tool is the best tool? Chris offers some guidance based on his real world experience as an industry analyst.
You can really know based on:
- Continuity with other systems
- It has an expected result/outcome
- It has a hard and/or soft ROI.
- You can know the total cost of ownership (TCO).
But sometimes the best tool is “the most okay tool.”
On the other hand, you can easily find yourself misled by:
- Open source. Companies may not realize that a commercial entity open-sources a part of their offering, but still has a commerical interest, subject to change. You also need to be aware of project activity, security, and following of OSS standard practices.
- Politics. If politics factors into the decision to acquire a tool, that’s an issue. For instance, if a new member of leadership is bringing in a tool from their past life or because they like someone there, you’re in a bad situation. Tools don’t solve people problems.
- Fairy dust. Vendors know their tools very well, but not your environment. This creates an expectation that a tool will solve all of your problems easily or bring you into the world of the latest and greatest. Demos are framing — not exactly what will happen in your environment.
Stages of Tool Adoption
What does tool adoption look like? Well, you’ve got three conceptual stages:
- Vetting. Vetting should include multi-role review, testing for compatibility with other tools, admin testing, and support/success testing.
- Procurement. You need to evaluate price and value and not delegate technical decisions to people in a procurement department. And think about renewal and future engagements.
- Implementation. This is another multi-role concern, and don’t let anyone hoard the tool. Go the opposite route and encourage demos and learning around the tool.
Decision-Making Resources
If you’re looking for help choosing tools or making recommendations, here are some resources you can make use of.
- Your Peers.
- Stack Overflow.
- Vendor Blogs/Documentation/Tutorials.
- Adopt an Advocate — follow developer advocates that you like, know, and trust.
- DevOps.com‘s market guide.
- SlashData.co: state of developer survey.
My Takeaway
As someone who has spent time in a lot of roles in the IT organization, I can relate to this from every angle. I have been (and, who am I kidding, probably remain) a developer demanding to have my favorite tool everywhere I go. I’ve also managed people clamoring for their favorite tools and found it counter-productive and tiring.
In tech, tools are important, of course. The right tools can be a game-changer and a competitive advantage. The wrong tools can sap the productivity right out of your organization. But what we often overlook are the superfluous tools and their cost.
In a world where we clearly can’t afford to wander around reinventing wheels, any engineering group needs to judiciously select tools. But we all have a mindset that drives us toward selecting them emotionally rather than judiciously. So companies need to develop a first class, objective framework for evaluating tools.
Search Fast Using First Principles, 3:00 Pacific
At the complete opposite end of the building, I arrived a little late to a talk by Steven Czerwinski, CTO for our friends at Scalyr.
This talk is a dive into how Scalyr handles search so quickly. Scalyr is a tool created by engineers, for engineers, that aims at making search of log files super fast and powerful.
When you’re thinking about fast search, you’d probably think naturally of indexing. Index is the default solution for search, in fact.
But as they were building Scalyr, they discovered that this wasn’t going to be the best way to go. It suffered from some problems:
- Storing indexes on disk is expensive, from a disk perspective.
- Code complexity of sync and the processing overhead of index maintenance.
- Indexes aren’t good at wildcards… or regexes… or ranges… or summarizing billions of matches.
So, counterintuitively, there were challenges and compelling reasons not to index. And so they opted not to go this route.
Brute Force FTW
Speaking of counter-intuitive, how about brute force? This is what they went with, and it has it place.
Consider grep, for instance. They used grep and its approach as the basis for cost-effective, high volume search, and they arrived at this approach by doing some math.
They set a goal of searching 1 day’s logs in 1 second for their customers. By starting with this goal, they were able to work backward from what mattered to customers and arrive at an approach.
Adapting with Users and Business Goals in Mind
This sort of reasoning helped them learn and make design decisions over the years. For instance, they’ve opted to use NoSQL, columnar database for storage. This approach is a lot more conducive to the searches that their customers typically do.
As they’ve evolved and scaled, they’ve kept a focus on what it is that their customers need. This system performs very well on search queries that their customers will run, which is important.
It doesn’t do as well for running the same queries episodically. So they have adapted and built a secondary system, instead, to answer known questions asked at a frequent interval. The gist of this approach is to index the queries, rather than the data.
My Takeaway
One of the things I find so cool about the approach and the business philosophy here is that it reflects a willingness to think without constraints. It’s easy to start any given project or venture with a lot of baked in assumptions — even when discarding them would open up a lot of creative possibilities.
“I’m going to solve this with brute force” isn’t just something you probably wouldn’t hear, but it might even invite scorn. And yet, it turns out to be a really creative way to come up with an approach that serves as a good business model and provides clients with what they really want.
So while we should all build on a basis of prior knowledge and bear that in mind, we shouldn’t be mesmerized by it to the point where we fail to explore all possibilities. I personally am something of a contrarian and iconoclast, so I disproportionately enjoy seeing this in action in the real world.
Commercializing a Major Open Source Project, 2:30 Pacific
This a talk by Andrew Roberts, CEO of Tiny, a popular open-source library for WYSIWYG/rich-content editing. WordPress, for instance, has used it.
TinyMCE was a purely open source project from 2004 to 2015. At that point, Andrew got involved with the project creators and looked to commercialize the product.
Tiny itself was founded in 1999 and called E-Fox. The company was reborn in 2015 as Tiny and raised some venture capital. Today, they have an engineering team in Brisbane, the original founders in Sweden, and some folks in other places.
Now for some interesting open source lessons.
Lesson 1: Free is not a Business Model
Matt Assay said that “entrepreneurs shouldn’t try to monetize open source, ever.” Tiny’s experience has offered a counter-example, but the point remains that it’s not easy to do it.
If you want to think about value propositions and how to make money, you can make use of a business model canvas. This gives you a way to think about how you’ll make money, from whom, and more besides. But, more importantly, it will let you brainstorm how to bring revenue into the system.
Lesson 2: Freemium is a Business Model
So how can you get revenue into the system?
Some people spend any amount of time to save money. Other people spend money to save time. It is that philosophical difference that makes the business model.
-Marten Mickos
Both types of people exist in businesses. You probably want those that will spend money to save time. By adding proprietary value to open source, you can target these folks.
To put it succinctly, proprietary value == the easy button.
But you really need to get the word out, because premium open source conversion rates are extremely low. Look for a big community.
Lesson 3: Ask for the Money
Traction is the rate at which you turn non-paying users into paying customers. This is difficult, so freemium companies employ a lot of tricks to gain traction.
- Powered by messages
- Email address collection
- In-App messaging
- Conversations
These things are tough to do and a lot of people shy away from them, but they’re necessary in order to gain traction. Don’t hesitate to call for upgrading.
Lesson 4: Have Clarity around the Business You Want
Think of the business of your dreams, and ask yourself about things like revenue goals, subscriptions, customer dependency. etc. You need to think about what you can do over the course of time to make things sustainable for you and to be effective.
All of the different business disciplines will probably require a team, so think through the team size.
You’ll also want to think about the number of buyers that you’ll need and how much you’ll charge each of them. Will you have a few million dollar whale clients? Millions of few-dollar clients?
My Takeaway
I wasn’t sure what to expect, exactly, going in. I was just generally curious about the model of taking a free and/or open source product and finding a way to commercialize it. I have no intent to do this, myself, but I find it interesting.
And my takeaway from listening to Andrew is that it’s rewarding, but you really have to (1) have a good, solid plan and (2) understand what you’re getting yourself into. I get the sense that he probably talks to a lot of folks with naive ideas about turning a project they love into their living. And I could certainly see that being a path to tragedy and disillusionment.
But Andrew’s path to success seems to suggest that one absolutely exists if you do formulate a good plan and you do undergo something of a commercial mindset shift.
Finding Product Market Fit, 2:00 Pacific
(As an appropos of nothing aside, I find myself wondering at the convention I established to fully spell out the time zone after each H2. I can only conclude that I did something dumb and then just kept doubling down for no good reason.)
This talk is by Sara Mauskopf, co-founder of Winnie. The entrepreneur in me couldn’t resist this, even though my main mission here was dev-centric talks.
Product-market fit is among the most important things for a founder to find. But how do you get there, and how do you know when you’re there?
She set out to build Winnie 4 years ago and took a winding path there. So she’s going to recount her experience.
Product/market fit means bieng ina good market with a product that can satisfy that market.
-Marc Andreesseen
In the Beginning: Tactic #1 Just Launch It
When they started with Winnie, the market they started with was parents. 90% of new parents were millennials and looking for info online. Millennials are not just parents, but they’re also “spendy.” Seems like a good market.
So, everything is easy, right? Just build a product for this market!
They started with what a lot of founders do: built an app for themselves. They built “Yelp for parents.” It was information just for parents, and they thought this was a good idea.
They built it, launched it, and evaluated product-market fit. And that’s tactic #1: just launch it.
A lot of people will dither and launch things in private beta and such. But they built it quickly, launched, and promoted it, getting featured in the app store. This got them a lot of users, which gave them the data to evaluate product market fit.
Up and to the Right, But Mistake #1
They did grow, but it was really hard. It required marketing stunts, growth hacks, and other non-scalable things. The growth wasn’t organic.
The mistake they made was to force it. So there wasn’t product market fit.
This potentially calls for a pivot, sometimes to a totally different market or product. In their case, they liked the market, but they didn’t throw out the product and go back to square 1. Instead, they employed tactic # 2.
Follow Net Promoter Score: Tactic # 2
Because their product was “Yelp for Parents,” it was a lot of things to a lot of people. But was there any one segment getting a lot of value? So much value that they were willing to promote the product?
To evaluate this, they segmented the user base into use cases.
What they found was that people who were using their product to find daycare and preschool were getting a lot value. This was the killer use case, and they learned it from the NPS data.
Scaling-Schmaling: Tactic #3
From here, they collected data manually, collecting data in just a single place — San Francisco. They spent money to do it.
Sara advises that, when you’re identifying product-market fit, don’t worry yet about scaling. Just collect the info, even if it requires disproportionate spending.
This went well. When they launched this in a mothers’ group in San Francisco, people were really excited. So excited, in fact, that they shared it unprompted, generating the organic promotion they were hoping for.
So did they have product market fit?
People Need to Be Paying for It: Mistake #2
The popularity might make it seem that way, but the money said otherwise. A lot of people love free things, but you can’t run a business purely on free.
So they started to think through who might pay for this service, and why. Would the daycares pay for it? The parents?
They advertised the products for sale and the measured clicks in order to evaluate. Both groups clicked, but the parents clicked a lot more.
So they decided to build a premium membership offering for parents, including payments infrastructure.
People Might Fib about What They’d Pay For: Mistake #3/Money Talks: Tactic #4
Unfortunately, it turned out that parents dropped out during the sign-up process, before giving their money. So they said they would by clicking, but they didn’t back it up.
So rather than polling them about what they would pay for, employ the Tim Ferris tactic of actually taking money. Get them to pre-order or sign up to evaluate wether they’d pay.
Sara then did something manual and easy. She set up square invoicing and tried this out with the daycares. She sold them a path tot the top of the search results, which they did manually.
And providers were absolutely willing to pay.
They’d pay for a month, and then they were more than willing to renew and keep paying to stay at the top of the results. Kids constantly age out of the programs, so the providers constantly need to be discovered.
So Sara added “AND make money” to the definition of product-market fit. You need a good market with a product that can satisfy that market, but you need people willing to pay.
My Takeaway
Toward the end of her talk, Sara said something that resonated with me, personally, as a business owner. She’s about to have a baby and she said that product-market fit, to her, had a trailing indicator of “can I step away from this business for a while, confident that the business would continue to be fine and even to grow.”
Last fall, my wife and I took our first vacation in a long time for, for over 2 weeks. And I had a very similar feeling. It was a testament to the viability of the business and our structure that stepping away for a while didn’t result in disaster or collapse.
If your presence and Herculean effort is a defining requirement of the business, then you have more of a job than a business.
Time Machines, Horror Stories, and the Future of Dev Tools, 1:30 Pacific
Once again, I came in a little late due to this being super far from where I was for the last session. This talk was by Shea Newton, of ActiveState.
The Hook
But, I got right into the recounting of a horror story. Two people, Perceptive and Dense, are in a car in the woods. Perceptive hears some scratching that Dense dismisses, but they both eventually notice it.
Eventually, they decide to leave. They hear on the radio that there’s an escaped convict in the woods, armed with a hook. When they get home, they find a hook in the car.
But where was this hook? Where did the convict get it? Why leave it?
The Ribbon
Then he talked about another such story: a couple, one of whom always wears a colored ribbon. Of course this comes up from time to time, but the woman who wears it always says “it’s just my thing.” Eventually on her death bed, her partner asks her to remove the ribbon and she agrees, at which point her head tumbles off.
Alright, but what kind of creature is this woman? How did this never come up? Anything that could be done about it?
Are there morals to these stories? If there are, it’s not really clear.
Tales of Developer Horror
Shea does relate them, however, to developer stories of horror. Here are some:
- The text parser that nuked his system. As he was working on creating a text parser, a library he’d neither heard of nor cared about eventually crashed his system and it just wouldn’t boot back up. He eventually got it sorted, but that ended his enthusiasm for the project.
- Python Game Development. He didn’t have experience with this and was running Linux while the product documentation was for Windows, but figured, “how hard can it be?” Pretty hard, it turned out. Even armed with Quora and Stack Overflow, nothing worked, even at the very bottom where the downvoted answers were. After a hard slog to get going, he asked himself what he could learn from this experience, and, like the old tales, there were no clear lessons. Give up sooner?
In none of these stories was the protagonist the problem. The root of the story is that building things is hard. But what can you do with that information?
Perhaps the answer lies in making the process and the build easier. The future of developer tools should really involve getting rid of these types of no-win conundrums and making things easier in the development world.
My Takeaway
I definitely relate to this. I can’t tell you how many late nights I’ve spent, defeated and tired by some set of snarled dependencies and concluding “there’s no real lesson to learn here other than that the universe can be cruel.”
I like the general concept that dev tools companies should look to minimize this dynamic in the world of software developers. I personally feel that there is a tendency in our community, to this day, to assume that RTFM is our responsibility and that the pain of assembly is just inevitable.
Developer experience is something that companies consider all too infrequently.
So I love the idea of someone actively striving to consider and solve this issue related to my experience. Just because I can spend all day figuring out a dependency snarl doesn’t mean that I want to or that I should.
Dunbar, Conway, Mythical Man Month, 1:00 Pacific
Steve Deasy, head of engineering at Atlassian, gave a talk about unleashing the potential of teams. He mentioned the idea of design patterns, and talked about applying similar concepts to organizations.
For instance, consider the pattern of a team growing. You wind up with drag factors: more planning/less doing, more meetings/less code, slower teams/reduced velocity. How do organizations fix this?
At Atlassian, the idea is what Steve calls “radical autonomy.” You want teams not to need central decision-making as much. Management can enable this by looking out ahead and seeing what sorts of external dependencies will become issues for the team. Then, you remove those as issues by preparation.
Like J.R.’s talk earlier, Steve talked about the Dunbar number and its importance. Atlassian’s growth has taken them past that number, making scale patterns more important. They’ve solved this by creating logical boundaries, within which smaller teams can have autonomy.
This has allowed them to scale through the limits of the Dunbar number while remaining efficient.
Pattern: Are You Shipping Your Org Chart?
As an introductory example, he asked if you’ve ever been to a website where clicking the “buy” button took you to a page that felt like a completely different experience. And he also cited the thing you hear internally as “why are different teams building the same thing within the company?”
This is, again, Conway’s Law, and it’s a familiar (anti pattern. But Steve talks about how you can actually turn this into a positive or a proper pattern.
He cited his work at Groupon as an example. As they grew, they were hitting constraints and inefficiency, and they were working to decompose a monolith. As this happened, the architects gave some thoughts about how to address this.
The team built an architecture that allowed many independent teams to work on front end components. This worked well because it involved reasoning up-front about where partitions would make sense in the software, and then structuring the org chart accordingly.
Pattern: Project is Running Late So Add Another Person
And now for the Mythical Man Month part of the title. Just about anyone with experience in the industry can relate to this.
Falling behind? Just add more people! It’s the wisdom that gives us the idea that nine women can have a baby in a month if waiting nine months for one to do it is too slow.
Steve talked about the tendency, upon falling behind, to “fire up the spreadsheet” and start talking about resources.
But, as most of us have learned through experience, this doesn’t actually work in practice. As Fred Brooks pointed out, “adding people to an already-late project will make it even later.”
But there’s some nuance here. Bear in mind, this applies when we’re talking about strangers — about people who are not familiar with the project.
You can get some mileage by bringing in people with relevant expertise or experience. Don’t just throw bodies at a problem — put the right people in the right seats.
My Takeaway
I found the theme of design patterns applied to the organization to be an interesting one. And I also enjoyed the wisdom of not taking established industry witticisms as complete canon.
We glean interesting insight from ideas like Conway’s Law or the Mythical Man Month, but those aren’t inviolable laws. There are ways that you can get something out of adding people to a late project or shipping your org chart.
So recognizing patterns and their implications as you grow becomes critically important. As someone who owns a rapidly-growing business and is very results/experimentation-driven, this message particularly resonates with me.
Learning Stuff at the Booths, 12:55 Pacific
I’ve been wandering around for the last hour, chatting booth attendees’ ears off, and trying to strike a balance between endless curiosity and not wasting their time, particularly if I don’t have a likely purchase case in the near term future.
Still, it’s tough. I get curious and my mind starts to explode with possibilities. Here’s the TIL rundown of some of the companies I talked to and had, “hey, that’s a thing!” moments.
- Corvid, by Wix. If you’ve always thought of Wix as “that DIY web platform for non-techies,” then, hey, me too. But, apparently, they’re remedying that by starting to create first class front-end development activities.
- Crafter Software is a company that provides an open source, headless content management system (CMS). Their commercial offering layers support and hosting atop the OSS product.
- TextControl is a document creation/management platform with roots in the .NET world. Seems like something I might be able to plug into our Hit Subscribe line of business software.
- I learned about the SWAMP: software assurance marketplace. As a static analysis aficionado, I love this. It’s a multi-language, multi-tool static analysis marketplace. Upload your code, select the analyzers you want to apply, and let ‘er rip. I do a lot of organizational consulting on the back of static analysis and a tool like this is SUPER handy for me.
Booth Touring, 11:55 Pacific
I’ve been strolling around a little since the last talk ended, checking out the booths. That’s after I retired to the lounge to refill on coffee, anyway.
I think the booth situation is probably familiar to anyone who has attended conferences in the past. I have nothing specific to report of interest on that front.
But I do personally enjoy strolling around and seeing what companies are up to. Historically, I might have enjoyed the swag angle, but my wife and I have been on a minimalist trek over the last few years, so there’s not a lot of appeal to collecting stuff. Nevertheless, I feel like booth strolling is a good opportunity to chip away at the problem of “you don’t know what you don’t know.”
I’m hoping to find a way to relieve pains I didn’t know I had.
Introduction to the Misty Robotics SDK, 10:30 Pacific
I couldn’t miss this talk. 20+ years ago, the main thing that drove me toward a CS degree was the promise of programming robots. And here’s a real, live robot.
This is a demonstration of the Misty Robotics SDK, by Justin Woo.
I wound up coming in a little late, because of difficulty finding the right stages, but I was immediately intrigued. He spun the robot around and showed an Arduino in her backside, and talked about her finger-print sensing capability, making her well suited for security.
He also cited some other good use cases for a small, friendly-demeanor robot:
- Education, innovation, experimentation
- Elder care
- Concierge services at hotels
Now for some live coding!
Demonstrating the API
Justin fired up a browser-based command center, and showed capabilities like changing her LED (in her chest) to different colors and having her play sounds. He also did a detailed walk through of the different sensors that she has that you can use.
From there, he introduced us to the idea that the robot is exposing her API as a REST endpoint. He showed a demo of sending a POST request to change Misty’s color.
(As an aside, Misty is very easy to anthromporphize and, dare I say, cute. She blinks expectantly at you and makes friendly sounds, and it kind of creates the feeling that she’s a smart, friendly pet.)
Justin then demonstrated that they’d integrated with Twilio to rig up something where you can send SMS messages that trigger certain REST endpoints. So he was sending text messages like “get sleepy” and she responded by doing it.
The last thing that he showed involved VS Code. The idea is to allow you to deploy code to Misty even when she’s not connected to the internet.
So he showed some Javascript code in the IDE that he could deploy directly from there to the robot. The code he demonstrated had her recognize a face and respond happily. So he deployed, spun her around, and she recognized him and seemed pleased.
I think that immediately makes Misty superior to my cats.
My Takeaway
I love this so much on a personal level, just because of my lifelong interest in the subject matter. I have no practical application that I can think of for Misty, but it’d sure be fun to have one and program her to truck around my house, smiling and applauding when I do mundane house chores.
Immediately, I looked up her price point, and it’s a non-trivial $3K, so I probably won’t get out my credit card in the immediate future. But I’d sure like to keep my eye on this for when I’m either a lot richer or the entry level price comes down.
But, if you work for, say, an app dev agency and you bring customers in to try to impress them, I think you could do worse than buying a few for your bench devs to play with. Seems like it’d be a great, attention-grabbing way to showcase the team’s ingenuity.
Save Weeks of Coding, 10:00 Pacific
This is a keynote, and I find myself in the large room with loud music and such. This is a talk by J.R. Jasperson of Twilio. His job, as chief architect, is to help Twilio scale.
Dunbar’s number: the limit to the number of people with whom one can maintain stable social relationships.
For humans, this number is, apparently, 148 people. And this matters because in context here because it’s a knee-point that creates a problem for agile software development.
Agile is, after all, about “individuals and interactions.” So this starts to break down.
Conway’s Law: organizations which design systems… are constrained to produce designs which are copies of the communication structure of those organizations. Or, as J.R. puts it, “you ship your org chart.”
Jasperson’s Law: You can make anything seem authoritative if you label it as a ‘law’ with someone’s name as a prefix.
(As an aside, that’s the best maxim I’ve read in a while)
Anyway, his point with an eponymous law is to suggest that efficiency doesn’t have to erode with growth. Scale doesn’t have to ruin your app dev process.
As an extreme org-scale skeptic, I’m intrigued as to where this will go and hoping to see a satisfying devil’s advocate case.
Efficiency Erodes Growth, Historically
With org scale, you tend to see strategic misalignment and tactical misalighnment, both.
As a vivid example, he showed a picture of a mansion in San Jose. It’s so big that it even has 2 basements. But it only has one working bathroom (and 12 “decoy” bathrooms). Sarah Winchester, heir to the Winchester Rifle company, commissioned this mansion.
When she acquired her wealth, she felt burdened by the means of acquisition, so, naturally she went to see a medium. The medium said that she could assuage the situation by continuously building her house.
Apart from being the craziest thing I’ve ever heard, it became a paragon of “confusing activity with progress.” It’s a vivid example of building without a plan.
J.R.’s point? The solution to this would have been a blueprint.
Blueprints at Twilio
A blueprint is a shared document that captures:
- Key problem to be solved.
- Requirements from stakeholders.
- High level designs focused on things that would be expensive to change later. The idea is to avoid boxing yourself in.
So, why do this?
Well::
- It drives alignment
- It’s a forcing function to think it through and discuss ideas with others
- Promotes best ideas and minimizes sprawl and mistakes
- Removes surface area for misinterpretation
- More eyes make for better plans.
In practice, blueprinting is a balancing act. Too little planning creates waste in the form of future rework, but too much planning re-creates all of the problems of waterfall software development (and Twilio is an agile shop). And, along those lines, J.R. stresses that the blueprint is a living document.
Blueprinting Lessons Learned
There were plenty of mistakes along the way, he says, that provided learning opportunities:
- Use a tool to manage blueprints
- Find weaknesses with sequence diagrams
- Remember rejected alternatives (it’s much easier to remember what you did than what you opted not to)
- Measure and iteratively improve: we want the process to serve us, not the other way around.
My Takeaway:
I was intrigued by this, particularly since Twilio’s group exists in a gap between my experience, which has been, historically:
- Working with small organizations that are nowhere near Dunbar’s number and
- Consulting at organizations that far surpass Dunbar’s number who are trying to figure out how to be more like (1)
It has been my experience that the road to scaling horribly is paved with “here’s how we scale sensibly,” but there’s admittedly an element of selection bias there. Organizations haven’t historically called me in as a management consultant and paid me to tell me about how great things are going. In other words, I’m almost by definition working with companies that have scaled poorly.
So it’ll be interesting to see how “sensible blueprinting” continues to scale for Twilio in the future. Good development process at scale is a tough nut to crack, so anyone cracking it deserves admiration.
What’s Holding Back Analytics at the Edge, 9:30 Pacific
I came in slightly late because I was wrong about which room the talk was in. Oops. And being late is worse today, since the sessions are only half an hour.
But here I am now.
The talk is by Tom Yates of Nubix (I think, anyway — there was a different speaker scheduled, so I did a little on-the-fly detective work on LinkedIn) and he was talking about the idea of “edges” in enterprise architecture:
- Server edge
- Device Edge
This talk is mostly about the device edge. For those of you following along at home, “edges” are entry points into enterprise systems. So, the company-issue smart phone, or your own smart phone with a company app.
“Software is eating the world, but only nibbling at the edge.”
Tom added a slide with this quote, and I enjoyed it.
Understandably, there are a lot of challenges when it comes to ovtaining and processing data from and related to these devices. Latency and processing time are factors.
There are opportunities here, though.
He cited Chevron as a case study. They take in 30 TB of data per day from wells, but they only analyze a small portion of it to help give them intelligence about changing wells and pump rates and whatnot. (I am not an oil man, but I can still imagine the conundrum)
Analytics Gap
One of the big challenges here is that Analytics folks work in Python and R, in cloud-based environments. But the people working directly with field firmware work with C and C++, so there are mundane logistical issues with extracting the data and getting it somewhere that the data folks can analyze it.
There’s also a mindset different with embedded and cloud folks. Embedded developers can brick things, so they’ll proceed more cautiously and update more judiciously, and rightfully so. But this makes it hard to instrument a lot of analytics collection capabilities.
My Takeaway
The entrepreneur in me immediately perceives a large opportunity here. I’ve been a home automation (pre-IoT) aficionado since the early 2000s, and have at least dabbled in all of the concerned techs: embedded systems, IoT, data analysis.
There is a TON of data out there being generated, and there’s a lot of waste happening as it just evaporates. Finding pragmatic ways to capture/analyze that information seems like it will be a prohibitive competitive differentiator in the not too distant future.
In other words, if you’re not using devices at the edge of your infrastructure to create actionable intelligence, you can bet your competitors will.
Hotel Room, 9:25 Pacific
Back at it on day 2! Yesterday was fun, and I meant to close things out with a summary of the day, but I got tired. Oh well, c’est la vie.
Had a leisurely catch-up on work this morning in the lounge along with my coffee and got ready for the day. Now, I’m running a little late, so time to hustle down for the talks.
I’m not wall to wall talks today, so I’ll try to share some personal notes/observations from the conference.
