FSTOCO 2014 - Combine everything

class: center, middle

# Combine everything
## Without dirty hacks

@BrianJGraham

---

# What I work on

* An educational Slideshow/Workshop
--

* With teams that also get their own slides
--

* Interactive forms ("games")
--

* Users have very high expectations
--

* Users are...
--

* High-level "decision makers"
--

* From Toronto to Beijing, Saudi Arabia to New Zealand.
--

* Behind really mean firewalls
--

* Even worse... Behind hotel WiFi

---

# Pre-refactoring

* Symfony1.0 powered
--

* Web Dashboard
--

* App loading
--

* Legacy Zend AMF code
  * Fetched directly from database
--

* Monolithic flash app
  * Configured by manually built XML
--

* The database was modified by external applications

* ... it worked!

---

# Today

* SF2 backend
--

* Transforms DB objects to JSON
--

* Builds XML file in compliance with backend logic
--

* Provides RESTful API endpoints
--

* Push JSON objects to redis queue
--

* Socket.io manages events
--

* Added a mobile app
--

* Added a back-office app
--

* Some business logic is still tightly coupled
--

* No monoliths
--

* Clear responsibilities!

---

# My focus today...

* Isolate applications
--

* Step-by-step
--

* Make contracts
--

* Demonstrate real-world problems
--

* Show how easy these things can be to solve

---

# Pre-refactoring

![Default-aligned image](old_dt.png)

* It worked... 
--

* Where's the model?
--

* How can we maintain this?

---

# Targeted elimination

![Default-aligned image](old_dt_remove.png)

* Remove dependency on external services
  * Remove voodoo
  * Remove CRM's DB

---

# Symfony2 Migration

* Establish basic model format in one place
--

* Transform data to JSON for polling

* Transform data to XML for Flash app config
--

* Remove external database manipulation; added API

---

# Symfony2 Migration
## Models

* Now models have a home; data is sourced from one place. Data is transformed into what is needed.

```PHP
class ApiController
{
    // Called one time on page load
    public function configurationAction( Presentation $pres )  {
        $content = $this->get('ep_api.transformer')
*            ->transformToXml('presentation_collection', $pres)
        ;

return new Response($content);
    }

// This method gets polled, remember me, I come up later in the presentation
    public function slideStateAction( Presentation $pres )  {
        $content = $this->get('ep_api.transformer')
*            ->transformToArray('slide_view', $pres)
        ;

return new Response($content);
    }
}
```

---

# Symfony2 Migration
## Transformers

```PHP
class SlideToArrayTransformer {

private $transformerContainer;

public function __construct($transformerContainer) { /***/ }

public function transform($pres) {
        $result = [];
        
        foreach ($pres->getSlide->getComponents() as $component) {
*            $content = $this
*                ->$transformerContainer
*                ->transformToArray($component->getName(), $component);

$result[] = $content;
        }
        
        return $result;
    }

}
```

---

# Sf2 Model Retrospective

* Good
  * One authority on what the model is

* Less good
  * Flash app isn't aware of new models; broken until updated

---

# Symfony2 Migration
## Add API

* Now the API delegates between product and CRM database

```PHP
class PrivateApiController {
    public function updateUsersAction(Request $request, Presentation $presentation) {
*        $presentation->updateUsers($request->getUsers());
    }
}

class Presentation {
    public function updateUsers($users) 
        foreach ($users as $user) {
           /*
              Business logic about 
              adding users to presentation
           */ 
        }
    }
}
```

---

# Back office integration

* Remove external database interaction

* API calls put changes to a presentation

---

# Back office code

## Data in (CRM sends to Product)

```php
class PrivateApiController
{
    public function updateTeamsAction(Request $request, Presentation $presentation)
    {
        $users = $this->get('ep_api.transformer')
            ->reverseTransformFromXml('user_collection', $request->getContent())
        ;

*       $presentation->setUsers($users);

return new Response(null, 204);
    }
```

---

# Back office code

## Data out (Product sends to CRM)

```php
*class DeleteUserCommand {

// ...

*    protected function send($userAccount, $presentation)
    {
        $httpClient->delete(
            $this->getUriTemplate( 'meaningful/uri/{account}/{presentation}', [
                account => $userAccount,
                presentation => $presentation
            ]);
        )
    }
}
```
---

# Back office code

## Data out (Product sends to CRM)

```PHP
class ApiController extends Controller
{

public function deleteTeamAction(Request $request, Presentation $presentation, Team $team)
    {
        $presentation->getTeams()->deleteTeam($team);

*        $this->get('ep_api.delete_users.crm')->send(
*            $this->getUser()->getAccount(),
*            $team
*        );
    // ...
    }
}
```

---

# Back office Retrospective

* Good
  * No more voodoo on the database, app determines what happens
  * Mock responses
  * Optional stand-alone mode, isolated from CRM
  * Can control contract changes without breakage

* Less good
  * Slower than raw SQL by a tiny amount
  * Was not an incramental ship, but perhaps it was better in this case?
  * Needed to do this on both sides, deploy two products at once.

---

# JavaScript gateway to ActionScript (Flash)

It used AMF, switched to a JS layer

* XHR polling
--

* API interface
--

* App interface
--

* Flash ExternalInterface

---

# JavaScript gateway to ActionScript (Flash)

It used AMF, switched to a JS layer

* XHR polling
* API interface
* App interface
* Flash ExternalInterface

```

---

# JS API layer

```javascript
    var applicationInterface = new AppInterface({
        apiInterface: new apiInterface(),
        flashElement: '{{ swfId }}'
        topic: new Topic('someSubscriptionTopic', '{{ url('sf2_endpoint') }}'),
        debug: {{ app.debug ? 'true' : 'false '}},
*        initializationValues: {{ initializationValues }}
    });
```
* Pass a config value to your app

---

# JS API layer

```javascript
    var applicationInterface = new AppInterface({
*        apiInterface: new apiInterface(),
        flashElement: '{{ swfId }}'
        topic: new Topic('someSubscriptionTopic', '{{ url('sf2_endpoint') }}'),
        debug: {{ app.debug ? 'true' : 'false '}},
        initializationValues: {{ initializationValues }}
    });
```

* The API can be changed

---

# JS APP layer

```javascript
*    var applicationInterface = new AppInterface({
        apiInterface: new apiInterface(),
        flashElement: '{{ swfId }}'
        topic: new Topic('someSubscriptionTopic', '{{ url('sf2_endpoint') }}'),
        debug: {{ app.debug ? 'true' : 'false '}},
        initializationValues: {{ initializationValues }}
    });

swfobject.embedSWF( // Third party flash loading library
        '{{ asset_flash('app.swf') }}',
        '{{ swfId }}',
        ...,
        {
          jsPublishEndpoint: function calledWhenFlashPushesData(data) {
*            applicationInterface.publish(data);
          }
        }
    );
```

* Flash ExternalInterface pushes commands to the App.

---

# JS API layer

App subscribes to API data and connects to gritty transport logic
```javascript
AppInterface.prototype._notifyInit = function(){
    // ...
*    this._apiInterface.subscribe(this._apiEndpoint)
}
```

---

# JS API layer

---

# JS API layer

App subscribes to API data and connects to gritty transport logic
```javascript
AppInterface.prototype._notifyInit = function(){
    // ...
    this._apiInterface.subscribe(this._apiEndpoint)
}
```
```javascript
XHRInterface.prototype.subscribe = function(endpoint){
    // ...
    this._XHRtransport.poll(endpoint);
}
```
Gritty transport logic
```javascript
XHRTransport.prototype.poll = function(endpoint) {
    this._request('GET', endpoint).then(function(){
*        // pass results to callback passed from the APP
    });
}
```

---

# JS API layer

App subscribes to API data and connects to gritty transport logic
```javascript
AppInterface.prototype._notifyInit = function(){
    // ...
*    this._apiInterface.subscribe(this._apiEndpoint)
}
```
```javascript
XHRInterface.prototype.subscribe = function(endpoint){
    // ...
*    this._XHRtransport.poll(endpoint);
}
```
Gritty transport logic
```javascript
XHRTransport.prototype.poll = function(endpoint) {
    this._request('GET', endpoint).then(function(){
*        // pass results to callback passed from the APP
    });
}
```
App pushes to Flash
```javascript
AppInterface.prototype._notify(command, data) {
*    this._el.flashElement({command: command, properties: data});
}
```

---

# JS API layer

App publishes data sent from flash

```javascript
appInterface.publish(data) // Flash calls this
```
App publishes to API
```javascript
AppInterface.prototype.publish = function(data) {
    this._apiInterface.publish(this._apiEndpoint, data)
}
```
Api Interface connects to gritty transport logic
```javascript
XHRInterface.prototype.publish = function(data) {
    this._XHRtransport._request('PUT', data)
}
```

---

# JS Layer Retrospective

* Good
  * Easier to change transports (websockets, mocks, etc)
  * Easy to test and develop on
  * Easy to tie in multiple apps and features (profilers, etc)
  * Can shim data formats in both directions

* Less good
  * Had to ship with previous backend change
  * Adds a new level of complexity

---

# Architecture

![Default-aligned image](dt_1.png)

---

# Performance

* Kind of slow

```PHP
class ApiController
{
    // This method gets polled, remember me from earlier?
    public function slideStateAction( Presentation $pres )  {
*        $content = $this->get('ep_api.transformer')
*            ->transformToArray('slide_view', $pres)
        ;

return new Response($content);
    }
}
```
---

# Performance boost!

* Added redis!

```PHP
class ApiController
{
    // This method gets polled, remember me, I come up later in the presentation
    public function slideStateAction( Presentation $pres )  {
*        $content = $this->get('ep_presentation_engine.cache_manager')
*            ->getView($pres);
        ;

return new Response($content);
    }
}
```

Change cache on write to database
```PHP
    public function updateSlideStateAction(Request $request, Presentation $pres)  {
        // ... logic saves changes to database

*        $content = $this->get('ep_presentation_engine.cache_manager')
*            ->setStale($pres)
        
    }
}
```

---

# Redis Retrospective

* Good
  * Faster responses
  * Lower server load
--

* Less good
  * COW API endpoint suffers from thread-like issues
    * Workers could have been used instead
    * Partial caches could have also been useful

---

# Architecture

![Default-aligned image](dt_2.png)

---

# Adding a mobile app

* Build a standalone mobile app (AngularJS)
--

* mock data providers
--

* Uses same configuration values as the flash app
--

* Symfony doesn't know about app logic, it just looks like an asset

---

# Mobile app code

```html
<body ng-apps="namespacedApp">
  <ng-view></ng-view>
*  <script src="{{ assets_common('js/app.js', 'namespacedapp') }}"></script>
  {# 
     assets_common is a custom twig call that extends
     assetic, you could just drop in the path
  #}

</body>
```

* Next, add packages.json to require the repo
--

* Lastly, add code to move the package into assets
--

* That's all I needed to do to integrate the app.

---

# Mobile Retrospective

* Good
  * Started before we knew where it would go
  * Easy to plug in

* Less Good
  * ???

---

# Architecture

![Default-aligned image](dt_3.png)

---

# Changing polling to WebSockets

* Tried PHP WAMP
--

, it crashed the server

* Decided to use the well-known Node.js and Socket.io
--

* Added the 150-ish lines of node.js and SF2 for authentication
--

* Switched out the JS XHR polling logic for a Socket.io client
--

* It just worked because the architecture supported it

---

# WebSockets Migration

```
7242a48  Changes polling code into websocket queue code.
f5e62b7  Removes XHR things.
ae78d08  Adds socket.io client
39bc101  added authentication for nodejs incoming connection
```
---

# WebSockets Migration

```
7242a48  Changes polling code into websocket queue code.
f5e62b7  Removes XHR things.
ae78d08  Adds socket.io client
*39bc101  added authentication for nodejs incoming connection
```

```PHP
class Authentication// ...
{
    // ...
    public function saveAuthentication(FilterResponseEvent $event)
    {
*        $this->redisClient->set($key, json_encode([
           'isAuthenticated' => // ... 
           'sessionId'       => //  ...
        ])
        //...
```
---

# WebSockets Migration

```
7242a48  Changes polling code into websocket queue code.
f5e62b7  Removes XHR things.
*ae78d08  Adds socket.io client
39bc101  added authentication for nodejs incoming connection
```

* Just adds the vendor to my dependency list

---

# WebSockets Migration

```
7242a48  Changes polling code into websocket queue code.
*f5e62b7  Removes XHR things.
ae78d08  Adds socket.io client
39bc101  added authentication for nodejs incoming connection
```

```
rm XHRTransport.js
rm XHRInterface.js
```
---

---

# WebSockets Migration

```
*7242a48  Changes polling code into websocket queue code.
f5e62b7  Removes XHR things.
ae78d08  Adds socket.io client
39bc101  added authentication for nodejs incoming connection
```

Before...
```JavaScript
*XHRTransport.prototype.connect = function(topic) {
   var self=this;

*   this._request('GET', topic).then(
      function onFulfilled(response) {
          if (response.status !== 304) { // because cache happens
              topic.executeCallback.bind(topic)(message);
          }
      });
};
```
After...
```JavaScript
*SocketTransport.prototype.connect = function(topic) {
   var socket = this._socket;
   
*   socket.on(topic.client, function(){
      topic.executeCallback.bind(topic)(message);
   });
};

```
---

# WebSockets Migration

```
*7242a48  Changes polling code into websocket queue code.
f5e62b7  Removes XHR things.
ae78d08  Adds socket.io client
39bc101  added authentication for nodejs incoming connection
```

```php
class QueuePusher {
    protected static $KEY_QUEUE = 'queue_name';
    protected $queueClient;

public function __construct($queueClient) { ... }

public function notify($data) {
        $this->queueClient->publish(
            self::$KEY_QUEUE, json_encode($data)
        );
    }
}
```
--

* Instead of building a cache, just push messages to the queue

---

# WebSockets Migration 
## Adding Node.js

Authenticate from data stored in redis

```JavaScrip
var phpSessionId = cookie.parse(handshakeData.headers.cookie)[config.session.name];

redis.get(
  config.redis.prefix + ':node_authentication:' + phpSessionId,
  function (error, result) {
    return deferred.resolve(result);
  }
);

deferred.then(
  function (sessionString) {
    var session;
    session = JSON.parse(sessionString);
    socket.handshake.sessionId = session.sessionId;
});
```

---

# WebSockets Migration 
## Adding Node.js

Push messages to subscribers

```JavaScript
redisClient.on('message', function (channel, rawMessage) {
  var message = JSON.parse(rawMessage);
  io.sockets
    .in('/sessions/' + message.sessionId)
    .emit(message.client, message.data);
});

io.on('connection', function (socket) {
  socket.join('/sessions/' + socket.handshake.sessionId, function (error) {};
  socket.emit('subscribe', socket.handshake.sessionId);
});

```

---

# WS Retrospective

* Good
  * Didn't need to touch any application logic anywhere 
  * Faster, better experience
  * Works for all clients (Flash + mobile)

* Less good
  * ???

---

# Architecture

![Default-aligned image](dt_4.png)

---

# Back office client view

* Use same technique for mobile app
  * back office view is just a client

---

# Back Office code

```php
<div ng-app="epEiSimpleClient" class="bootstrap-styles">

*   <script src="{{ asset_common('js/app.js', 'epeisimpleclient')}}"></script>
    <link rel="stylesheet" href="{{ asset_common('css/app.css', 'epeisimpleclient')}}" />
    <script>
        angular.module('epEiSimpleClient').run(['$rootScope', function($rootScope) {
*           $rootScope.configuration = {{ configJSON|raw }};
        }]);
    </script>
</div>
```
--
* Same thing as the angular mobile app

---

# Back Office retrospective

* Good
  * Super easy to plug in
  * I only had to think about just that app

* Less Good
  * ???

---

# Architecture

![Default-aligned image](dt_5.png)

---

# Separation of concerns

What do I have to think about when building most of the app?

When do I break backward compatibility?

![Default-aligned image](dt_5_concerns.png)

---

# Repetition, repetition, repetition 
## Isolate, isolate, isolate!

* Isolate apps from input

* Abstract input

---

class: center, middle

# Thanks! :D 
<br/>

## Isolate everything
## Build one thing step-by-step
## Think about contract breaking

<br/>

### @BrianJGraham