How do you do to spell relief?

How do you handle Sev 1 critical outages?

Stay calm

Sev 1 outages are stressful. When production is down and customers are affected and everyone is looking at your team for an answer, first and foremost stay calm… breath. This is easier said than done when you are affecting millions of dollars per hour in transactions (which is a thing in large scale payment systems, not fun), but regardless of the impact of the outage, if you loose your cool, the solution can be sitting right in front of your face and you won’t see it. Shit happens, there will always be bugs, sites will always go down at some point, accept it, find the solution and focus on not letting the same shit happen twice.

Don’t focus on blame

Establishing who caused the issue is not important? Knowing who may have been involved in the changes that led up to an issue and made changes after the issue is important to understand. Even if there may have only been one person involved, you can’t assume that they are the cause and it does no good to blame anyone when production is down. Focus on the solution.

Create time line of events

You should document all relevant changes that led up to the outage and all changes that occurred after the outage. This not only helps to discover possible causes it provides documentation that can be used during root cause analysis and investigations during similar outages.

The time line can be kept on an internal team wiki so the team has visibility and can add to it as necessary. During an outage, someone should be assigned to record all of the facts in the time line. Without facts your poking at the problem in the dark.

Never theorize before you have data. Invariably, you end up twisting facts to suit theories, instead of theories to suit facts.
Sherlock Holmes to Watson (Movie – Sherlock Holmes 2009)

Investigate logs

I can’t tell you how many times that checking the logs first would have saved a lot of time as opposed to just poking around looking for file changes, config changes… everything else, but simply looking at the logs. The first step in investigating the issue should be looking at all of the relevant logs: event logs, custom application logs…

Communication is key

Keep a bridge line open with team

Keeping a bridge line open, even if there is nothing to discuss, keeps a real-time line of communication open and ready when someone has questions, ideas, and possible solutions.

Send regular status updates to team and stakeholders

Sending a message to announce the issue and what is known right now is good form. It lets everyone know that you are on top of the issue and working hard to solve it. If you haven’t found a resolution in a certain amount of time, sending another update explaining what has been done and any new findings lets everyone know that although you haven’t found the issue, you are still working hard on it. It may be a good idea to even post the status updates to a blog or Twitter, syndicate the updates to as many channels as you can, especially if you have a large application with many users.

Staying proactive with communication is much better than constantly having to field random calls and emails looking for information you should be readily sharing. Keep communications open and don’t try to hide, spin, or lie about the mistake.

No one makes any changes without discussing the change

While everyone is trying to solve the issue, no one should be making changes in production, even if the fix is blatantly obvious. A Sev 1 is serious and everything changed to fix it should be discussed with the team first so it can be documented and controls put in place to prevent it in the future.

If the team agrees on the change then the change should be documented on the timeline and a notification should be sent when the change is starting and when the change is finished. The change discussion and notifications can be simply talking it out over the bridge line or an IM or email. The point is don’t allow the change to get worse or be repeated by making undocumented changes that the team can’t learn from.

 

Conclusion

These are just some tips that I have learned over the years. I have seen many more sound practices, but the gist is:

  • Stay calm
  • Document changes to production
  • Work as a team
  • Learn from failure

Stand Up, Sit Down, Just Collaborate

I haven’t blogged in a while. I started a new job and I’ve just been busy getting acquainted. Well at the new gig we have been exploring ways to improve as a team. Stand ups have been brought up a few times.

I Hate Stand Ups

In my experience stand ups have been over rated status meetings. Maybe that’s because I haven’t been on a team doing scrum correctly or maybe standups are dumb, I’m not sure. I have done standups for years on multiple teams and there was waste in every one of them. There were some standups that were too big and ended up with a bunch of people with blank stares because most of the standup was irrelevant to them. There were some that were too small and redundant because the team collaborated closely throughout the day, so nothing new in standup. There were some that were unwieldly and not focus on standard scrum standup reporting and just a big waste of time. In most, there have rarely been instances where blocks were unblocked or time not wasted.

In all of these stand ups I had the feeling that there must be a better way than doing a thing in a religious fashion just because some abstract process that the community follows like sheep says so.

I envision a life without standups with a process that provides easy ongoing reporting of project status and impediments. Discussing project status is important for determining if a project is on schedule and if the financial goals of the project will be met. Discussing impediments are important because resolving issues that are blocking tasks will help prevent the project schedule from being in jeopardy and helps reduce frustration.

There is usually someone that is responsible for project status reporting. This person can make daily rounds and collaborate with the team on the collection of regular status and to work through some of the project impediments. I say some because some impediments are technical and require another team member to solve. Anyway, all of this could be done in a status meeting, but status meetings whether they are sit down or stand up have a tendency to turnout just like my experience with standups. One-on-one collaboration on status whether it is face-to-face, IM, or phone may be extra work, but I believe it provides more value in terms of time and effectiveness for the team. If you want to keep it srum’ish, the status walk through can be time boxed to a certain amount of time per person.

We addressed project impediments, but what about technical ones. We are a team, why can’t we just collaborate? Most likely you have an idea of who to ask for technical help and if you don’t you know how to reach out to your team to find out who can help. Go talk to someone. If you can’t ask your teammates for help for whatever reason, your team has a problem. I have been on teams with people who were closed to answering questions, but this is a new agile day. If you don’t want to help the team, you need to go solo. You don’t deserve to benefit from the fruits of the team’s labor if you don’t want to contribute to the team. If you are scared to ask for help, get over it. When you are wasting time on something that can be solved by just asking someone on the team is a disservice to your team. Taking a little time to discuss a problem and get pointed in a new direction is not an indictment of your skills. It’s a reason for you to collaborate and become a more cohesive team. This is not to say that you shouldn’t put in your due diligence or pound the same person with multiple questions day in day out. Become a nuisance and you may get voted off the island.

I guess my point is, you can be agile without scrum, you can do scrum without standups. If a process isn’t working, improve it.

OK, end rant… for now.

Part 7: Get Up and Running with #React and #TypeScript

The “Get Up and Running with React and TypeScript” series is made up of posts from chapters in my book “Hello React and TypeScript“. If you have questions, comments or corrections, please reach out to me in the comments or on Twitter @charleslbryant.

You can view the posts in this series on the Hello React and TypeScript category page.

Component Composition

https://charleslbryant.gitbooks.io/hello-react-and-typescript/content/Samples/ComponentComposition.html

This sample gives a basic example of refactoring the Hello World application by composing the UI with modular components.

Source Code

https://github.com/charleslbryant/hello-react-and-typescript/releases/tag/0.0.7

src/helloworld.tsx

/// <reference path="../typings/tsd.d.ts" />

import * as React from 'react';
import HelloForm from 'helloform';
import HelloContent from 'hellocontent';

export default class HelloWorld extends React.Component<any, any> {
    constructor(props: any){
        super(props);
        this.state = { name: this.props.defaultName };
        this.handleChange = this.handleChange.bind(this)
    }

    public handleChange(event: any) : void {

        this.setState({ name: event.target.value });
    }

    public render() {
        return (
            <div>
                <HelloForm 
                    name = { this.state.name }
                    handleChange = { this.handleChange } 
                />
                <HelloContent 
                    name = { this.state.name }
                />
            </div>
        );
    }
}

The HelloWorld component is updated to compose the same UI as the previous example using two modular components, HelloForm and HelloContent.


import * as React from 'react';
import HelloForm from 'helloform';
import HelloContent from 'hellocontent';

To compose with components the components you want to compose have to be in scope. To do this we import the components. HelloForm and HelloContent are imported by using the import statement with the from value being the name of the component.


public render() {
    return (
        <div>
            <HelloForm 
                name = { this.state.name }
                handleChange = { this.handleChange } 
            />
            <HelloContent 
                name = { this.state.name }
            />
        </div>
    );
}

The actual composition occurs in the render method. We add the HelloForm and HelloContentcomponents. Notice that the name of the components start with uppercase to differentiate them from HTML elements. Each of the components accept some properties and we pass them with a syntax that is similar to defining HTML attributes.


By composing UIs in this manner we move from an imperative style of building UIs to a declarative one. Instead of defining every element and attribute we want to use in the UI we delegate the definition to a modular reusable component.

Since the modular components are reusable we can compose multiple UIs with them. We can compose with components developed by other teams. We get to focus on the unique aspects of our domain and delegate other lower level concerns to modular components.

src/helloform.tsx

/// <reference path="../typings/tsd.d.ts" />

import * as React from 'react';

export default class HelloForm extends React.Component<any, any> {
    constructor(props: any){
        super(props);
    }

    public render() {
        return (
            <div>
                <input 
                    value={ this.props.name }
                    onChange={ e => this.props.handleChange(e) }
                />
            </div>
        );
    }
}

Here we have a new component responsible for the collecting user input for our tiny application. By now this code should be familiar. We just moved the input element from the HelloWorld component and encapsulated it in this new component.

For this simple example we are storing all of our components in one folder. As your application grows you may want to use a folder structure that makes the number of component files more manageable. When you are at the point of doing this you will have to address the reference path for the TypeScript typings. It will become a maintenance issue having to keep the path in sync as you move components and build new folder structures, but we won’t go into that just yet.

src/hellocontent.tsx

/// <reference path="../typings/tsd.d.ts" />

import * as React from 'react';

export default class HelloContent extends React.Component<any, any> {
    constructor(props: any){
        super(props);
    }

    public render() {
        return (
            <div>
                Hello { this.props.name }!
            </div>
        );
    }
}

This is a component responsible for displaying the hello message. Again, this component is a result of encapsulating this section of the DOM out of the HelloWorld component.

Stateless Components

One interesting observation about these components is that they are stateless. They don’t hold any state and they rely on a parent component to pass props to it in order to do its work. This includes event handlers and data as you can see in the sample code.

When possible using stateless components are preferred in React. They help to improve performance because they lessen the amount of processing that React has to do with the component.

Part 6: Get Up and Running with #React and #TypeScript

The “Get Up and Running with React and TypeScript” series is made up of posts from chapters in my book “Hello React and TypeScript“. If you have questions, comments or corrections, please reach out to me in the comments or on Twitter @charleslbryant.

You can view the posts in this series on the Hello React and TypeScript category page.

Accept User Input

This sample gives an basic example of accepting user input.

Source Code

https://github.com/charleslbryant/hello-react-and-typescript/releases/tag/0.0.6

src/helloworld.tsx

/// <reference path="../typings/tsd.d.ts" />

import * as React from 'react';

export default class HelloWorld extends React.Component<any, any> {
    constructor(props: any){
        super(props);
        this.state = { name: this.props.defaultName };
    }

    public handleOnChange(event: any) : void {
        this.setState({ name: event.target.value });
    }

    public render() {
        return (
            <div>
                <div>
                    <input 
                        onChange={ e => this.handleOnChange(e) }
                    />
                </div>
                <div>
                    Hello { this.state.name }!
                </div>
            </div>
        );
    }
}

The significant changes here are we removed the button and handleOnClick method. We also added a text box and a handleOnChange method to handle changes as a user adds input to the text box.


public handleOnChange(event: any) : void {
    this.setState({ name: event.target.value });
}

This is a new method to handle the onChange event. In the method we are calling this.setState and updating the value of name in this.state to the value of the target element passed in the event.


public render() {
    return (
        <div>
            <div>
                <input 
                    onChange={ e => this.handleOnChange(e) }
                />
            </div>
            <div>
                Hello { this.state.name }!
            </div>
        </div>
    );
}

Here we added an input element and set its default value to this.props.name. We also bound the text box’s onChange event to handleOnChange method.


With these changes we have implemented a unidirectional data flow.

  1. When the user enters something in the text box it triggers the onChange event.
  2. The onChange event is handled by the handleOnChange method.
  3. The handleOnChange method updates the value of name in this.state and triggers a re-render of the component with this.setState.
  4. this.setState ends in a call to the render method that updates the name in our “Hello” message.

State is only changes as the result of an event. The Hello message is no bound to an external model state and can only be updated as a result of an event being triggered by user input. This is different than two way binding or bi-directional data flow where changes in a model can also update the state of a view.

Unidirectional Data Flow (UDF)

event > event handler > state > render

Components are representations of the state of a view over time. As events are triggered over time they update state and re-render the component with the new state. The flow can be seen as a stream of events that flow in one direction that eventually update component state causing a component to re-render. If you know about CQRS, event streaming, or stream processing, there are similar concepts in each. UDF is a redundant theme in learning React, hence a redundant theme in this book.

The sample is a simple naive example because we aren’t dealing with external or persisted data. The scope of the example makes it a little hard to understand UDF. In the example we don’t have to worry about updating an external store.

If you are having trouble understanding UDF, when you learn about Flux it will makes more sense. The Flux architecture helps you visualize data flow in a circular one way round trip. Even though it may be hard to see UDF within the context of a single component the same event flow is used to accomplish UDFin React whether within a single component, a Flux architecture, or using Relay (another Facebook library). When you get into Flux or other data flow patterns or libraries, UDF will be expanded to add additional concepts into the data flow.

UDF vs Bi-directional Data Flow

The important thing to notice about UDF is that we aren’t attempting to create a complex bi-directional view binding with some external model. We are binding to the state modeled within a component with no dependency or complex mapping to some external data model. The component is responsible for expressing its own state, updating its state, passing properties to its child components, and re-rendering itself and children when state changes as the result of some event.

If we were to use a bi-directional binding to an external model, we would not know why state is being updated. Any number of views could have the same binding to the same model. With bi-directional binding a change to a view or model could cause updates to multiple models or views and it becomes hard to understand the data flow, especially when you are trying to solve a Sev1 incident.

If you’d like to know more about UDF, there is a lot about it online that can be found with a simple search. Actually, the original MVC pattern is an example of UDF. It was distorted when it moved to web clients. If you’d like to dig into the theory behind React and UDF, you can research Functional Reactive Programming – https://en.wikipedia.org/wiki/Reactive_programming.

Part 5: Get Up and Running with #React and #TypeScript

The “Get Up and Running with React and TypeScript” series is made up of posts from chapters in my book “Hello React and TypeScript“. If you have questions, comments or corrections, please reach out to me in the comments or on Twitter @charleslbryant.

You can view the posts in this series on the Hello React and TypeScript category page.

Component Interactivity

This sample gives an basic example of using adding interactivity to a component.

Source Code

https://github.com/charleslbryant/hello-react-and-typescript/releases/tag/0.0.5

src/helloworld.tsx

/// <reference path="../typings/tsd.d.ts" />

import * as React from 'react';

export default class HelloWorld extends React.Component<any, any> {
    constructor(props: any){
        super(props);
        this.state = { name: this.props.defaultName };
    }

    public handleChange(event: any) : void {
        this.setState({ name: "Charles" });
    }

    public render() {
        return (
            <div>
                Hello { this.state.name }!
                <button 
                    name = "Update"
                    onClick = { e => this.handleOnClic(e) }
                >Update</button>
            </div>
        );
    }
}

To add interactivity we:

  • added an element for a user to interact with
  • created a method to handle an event triggered by user interaction
  • bound the new element’s onClick event to the new event handler method

public handleOnClick(event: any) : void {
    this.setState({ name: "Charles" });
}

This is a new method to handle the click event. We expect callers of this method to send the event as an argument so we defined the argument as type any so we can accept any event. We could define a specific or generic event type and restrict the argument to that type. We also define the method as returning void or nothing.

In the method we are calling this.setState and updating the value of name. When we callthis.setState it causes the component to re-render with the new state.


return (
    <div>
        Hello { this.state.name }!
        <button 
            name = "Update"
            onClick = { e => this.handleOnClick(e) }
        >Update</button>
    </div>
);

We are adding a new HTML element, button (remember that HTML elements start with lowercase letter). The interesting bit is that we are binding the element’s onClick event to the handleOnClick method. To bind the event handler with the event we are using the new ES6 arrow function expression to simplify function context binding. Notice that the event handler is camelCased which is consistent with JavaScript style.

Function Context Binding

In the end, React is just plain JavaScript. The React Team has resisted the urge to add a lot of magic and have made strides to remove magic to reduce coupling. Magic is the encapsulated work that is hidden behind abstractions. When a library does work behind the scenes without you knowing it, its magic. Reducing this hidden work helps to losen coupling and that is a good thing because it enables reuse. The React team believes our code shouldn’t have a lot of coupling to React.

One example of this, related to our use of the arrow function above, is function context binding. There was a time when React automatically bound functions to this. If you aren’t a JavaScript developer there is a chance that you don’t know that this in JavaScript is different from this in C# or Java or self in Ruby.

In some strongly typed languages, this is bound to an object instance. In JavaScript this is bound to the function it is called in. This causes a lot of confusion, but this decoupling of this from their parent function allows us to reuse functions outside of the parent, powerful and dangerous at the same time.

To get this function reuse we can define the context for a function and have this represent what we want it to. In our example the arrow function is assigning this from the class HelloWorld to thehandleOnClick method. If we wanted, we could use this same method in say a GoodByeWorld class and the arrow function could set the context of the handleOnClick method.

Without the arrow function we would use JavaScripts bind method to achieve the same effect. The arrow function is just some syntactic sugar for

onClick = { handleOnClick().bind(this) }

You can get more on arrow functions here https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/Arrow_functions.

Part 4: Get Up and Running with #React and #TypeScript

The “Get Up and Running with React and TypeScript” series is made up of posts from chapters in my book “Hello React and TypeScript“. If you have questions, comments or corrections, please reach out to me in the comments or on Twitter @charleslbryant.

You can view the posts in this series on the Hello React and TypeScript category page.

Component Props and State

https://charleslbryant.gitbooks.io/hello-react-and-typescript/content/Samples/ComponentPropsAndState.html

This sample gives an basic example of using props and state in your components.

Source Code

https://github.com/charleslbryant/hello-react-and-typescript/releases/tag/0.0.4

src/main.tsx

/// <reference path="../typings/tsd.d.ts" />

import * as React from 'react';
import * as DOM from 'react-dom';
import HelloWorld from './helloworld';

const root = document.getElementById('app');

class Main extends React.Component<any, any> {
    constructor(props: any){
        super(props);
    }

    public render() {
        return (
            <div>
              <HelloWorld defaultName='World' />
            </div>
        );
    }
}

DOM.render(<Main />, root);

The change to main.tsx is minor. Here is the one change.


return (
    <div>
        <HelloWorld defaultName='World' />
    </div>
);

We are just passing defaultName to the HelloWorld component. This is doing exactly what you think it is, its setting the default value for who we are saying Hello to. Notice that this name is explicit in defining this input. It is a default value, the HelloWorld component can change this value and the assumption is that it may be changed.

Another thing to notice is the camelCasing.

All DOM properties and attributes (including event handlers) should be camelCased to be consistent with standard JavaScript style. https://facebook.github.io/react/docs/dom-differences.html

src/helloworld.tsx

/// <reference path="../typings/tsd.d.ts" />

import * as React from 'react';

export default class HelloWorld extends React.Component<any, any> {
    constructor(props: any){
        super(props);
        this.state = { name: this.props.defaultName };
    }

    public render() {
        return (
            <div>
                Hello { this.state.name }!
            </div>
        );
    }
}

We made two changes to helloworld.tsx, but they are significant to the interactivity of our little application.


constructor(props: any){
    super(props);
    this.state = { name: this.props.defaultName };
}

In the constructor we are setting the initial state for this component. this.state is a plain JavaScript object. It is a mutable representation of the model for the view. this.state is expected to change over time as events occur in the component. In this sample we are initializing this.state with an object literal{ name: this.props.defaultName }.

this.props.defaultName was passed in from main.tsx. this.props is also a plain JavaScript object. The difference being it is an immutable representation data used by the component. this.props should not be changed and they are passed in by parent components. If you change props, they will be overwritten when the parent rerenders. Mutating props puts your component in an inconsisten state, so don’t do it.


return (
   <div>Hello { this.state.name }!</div>
);

The next change is to the return value of the render method. We are using this.state.name to set the name dispayed in the UI.

In this sample we use this.props to initialize this.state. We know that we want to update the name of who we say hello to, so we are using a state object to represent the name. If we didn’t want to update name, we could use props directly. For example

return (
    <div>Hello { this.props.name }!</div>
);

Using props like this is saying that we don’t expect this component to change the name. In fact, to make it explicit, we changed the name from defaultName to name. Also, main.txt would have to be changed from passing defaultName to name. Using props means we expect the parent to be in control of changing the value of props. Using state means we expect the component to be in control of its own state.

Pretty URLs in Aurelia

I went to a local MeetUp on Single Page Applications using Aurelia and TypeScript. As I sat there soaking up this sweet framework, I was wondering about the hashes (#) in the URL of the demo application. There is nothing wrong with hashes besides being an unfamiliar nuisance to the uninitiated user looking at the funny characters in the not so pretty URL. I still wanted to eradicate them.

Drama

Actually, there was a big fuss over URL hashes when Twitter and Facebook popularized their use and Google decided to index hashed URLs. There was a popular theme going on how hash bang breaks RFC 3986 and other “stop hacking my URLs” nonsense, but the hashes worked then and are still here today.

Solutions

HTML 5 offered up a solution to get rid of them, but older browsers aren’t HTML 5 compliant. Older browser these days is code word for IE9 and older.

Some benefits of not using hash routing are

  • Future proofing SEO, Google deprecated hash URLs because of the HTML 5 standard.
  • Isomorphic applications and server side rendering, hash only work with JavaScript in client side routing.

Yet, there is some weird comfort in knowing that I have the power to ditch the hash in my single page applications if I want to.

In some routing modules you get to choose between hash and pushstate routing. Hash routing uses window.location.hash. Pushstate uses the new History API in HTML 5, history.pushstate. I figured there has to be something similar in Aurelia, right?

Even though it will kill some IE browsers, I still want to know how to do pushstate in Aurelia, what if I don’t care about older browsers. After a little digging on the Aurelia GitHub repos, I found issue #16 and #225 on router and issue #21 on history-browser that seems to address this very thing. Now that I was so dirty after all of that digging, I also went the sensible developer route and just checked the documentation, it has a section on Configuring PushState. So, it seems setting pushState to true turns pushState on (duh) and I’m still not sure about what hashChange does.

I know this is bad form for a blog post, but I haven’t tried this. I am assuming that router can be configured by setting options for pushState and hashChange based on the research above.

So, I am posing this as something for me to try or for someone to help me set it right.

configureRouter(config) {
config.title = 'Hello World';
config.options.pushState = true;
config.options.hashChange = false;
   ...
}

There You Go

So, you seem to have options for Pretty URLs in Aurelia.