Series – Part 3: Serverless Architecture – a practical implementation: Serverless REST API

In part two of this series I discussed creating a serverless data collection and processing fabric for an IoT deployment. To recap, we’ve now reviewed the local devices and controller/gateway pattern for the security cameras deployed. We’ve also discussed the Amazon Web Services infrastructure deployed to collect, process and catalog the data generated by the security cameras.

In this post we will cover the creation of a serverless REST API.

Continue reading “Series – Part 3: Serverless Architecture – a practical implementation: Serverless REST API”

Series – Part 2: Serverless Architecture – a practical implementation: IoT Device data collection, processing and user interface.

In part one of this series I briefly discussed the purpose of the application to be built and reviewed the IoT local controller & gateway pattern I’ve deployed. To recap, I have a series of IP cameras deployed and configured to send (via FTP) images and videos to a central controller (RaspberryPI 3 Model B). The controller processes those files as they arrive and pushes them to Amazon S3. The code for the controller process can be found on GitHub.

In this post we will move on to the serverless processing of the videos when they arrive in S3.

Continue reading “Series – Part 2: Serverless Architecture – a practical implementation: IoT Device data collection, processing and user interface.”

Series – Part 1: Serverless Architecture – a practical implementation: IoT Device data collection, processing and user interface.

 

reinvent_launch-page_illustration_lambda
AWS Lambda

Serverless architectures are getting a lot of attention lately – and for good reason. I won’t rehash the definition of the architecture because Mike Roberts did a fine (and exhaustive) job over at MartinFowler.com.

However, practical illustrations of patterns and implementations are exceptionally hard to find. This series of posts will attempt to close that gap by providing both the purpose, design and implementation of a complete serverless application on Amazon Web Services.

Part 1 – The setup…

Every application needs a reason to exist – so before we dive into the patterns and implementation we should first discuss the purpose of the application.

Nest wants how much for cloud storage and playback?

I have 14 security cameras deployed, each captures video and still images when motion is detected. These videos and images are stored on premises – but getting them to “the cloud” is a must have – after all if someone breaks in and takes the drive they are stored on all the evidence is gone.

If I were to swap all of the cameras out for Nest cameras cloud storage and playback would cost $2250/year – clearly this can be done cheaper… so…

Continue reading “Series – Part 1: Serverless Architecture – a practical implementation: IoT Device data collection, processing and user interface.”

Leadership: Risk or Opportunity Focused

imagesAs I’ve worked with teams engineering teams big and small – in both enterprise and startup contexts – over the last 20 years I’ve noticed two distinct patterns in leadership and their impact on the culture and productivity of those teams.

Pattern: Risk Focused Leadership

Risk focused leadership emphasizes the up front identification and mitigation of risk in any program or project. It attempts to know as much as possible before committing and rewards engineers who can identify and articulate risks.

Impact on Culture

Since the engineer’s perceived worth is derived from her ability to identify reasons things won’t work – or more precisely to avoid mistakes – the culture tends to favor inaction and exhaustive research and analysis.

Impact on Productivity

Predictably, these teams tend to have low output. Generally the output they do generate is both expensive and highly reliable. In enterprise contexts there tends to be a reliance on proven vendors – usually with a bias toward those with long market histories which can be analyzed.

Pattern: Opportunity Focused Leadership

Opportunity focused leadership emphasizes the potential gain of any program or project. It – often aggressively – attempts to capture opportunities as they present themselves. These leaders reward engineers who can grasp the opportunity and rapidly implement solutions which might capture the opportunity.

Impact on Culture

Since the engineer’s perceived worth is derived from her ability to create solutions which may capture the opportunity – or more precisely move quickly with imperfect information – the culture tends to favor rapid cycles of activity and an ability to “change gears” rapidly.

Impact on Productivity

Predictably, these teams tend to have very high output, however, much of that output goes unused. Generally – but not always – the output is proof of concept quality with a bias toward open source tools, frameworks and platforms. Since the long term viability of the opportunity and feature/product were not exhaustively analyzed teams learn to implement low cost solutions which can be “thrown away”.

Leadership Lessons

What should be obvious by now is that neither is bad or good – each is appropriate in certain contexts and, more often than not, a project, program or organization requires a well defined, understood and articulated balance between the two leadership focuses.

Some leaders are naturally opportunistic, some are risk managers. As a leader of a engineering or product development team it is your responsibility to understand:

  • The opportunity/risk profile of the program/project/product.
  • The relative opportunity/risk propensities of your team.

Most importantly, you must ensure that the opportunity/risk profile is articulated and the stakeholders understand and agree with the inherent tradeoffs for any program, project or organization. Failing to do that is the unacceptable risk you must avoid at all costs.