Low Latency Programing

Looks like their thinking are influenced by Peter Lowery' s Talk on Friday and possibly by
 Martin Fowler and Martin Thompson's writing about Low Latency.

I think there are two things highly relevant to Insight-Desktop (Carol, Hai, Daniel, Alec also in Peter's talk so please comment)

(1) Journaling, aka EventSourcing
   The idea is to log every input event and replay them to help debug PROD issue in DEV. Obviously, we need a low latency logger

(2) Profile Market watch.
   I think Market Data come in here. We should profile for GC pause, Lock contentioin, Caching.
   This help us to understand MVVM + Presenter pattern better in term of low latency,

Here are key points from their talk/writing

Setup correct performance test --- Theory are most likely wrong.
GC-Free  ---The biggest Performance cost is GC Pause
Lock-Free  ---- Lock cause Context switching, clear cache line
Cache Friendly --- L3 Cache is shared memory cross cores.
EventSourcing --- Replay input event to debug PROD in DEV, instead of analysing Log files

Actionable Items:

(1) Performance Benchmark Test App: 
    A WPF Unit testing simulator can be build for Logging, Journaling, Rx vs. .Net event, Market data <100 micro-second.

(2) Logging Improvement:
    a separate email thread already starting writing a logging using RingBuffer.

    



Shared Memory (L3 cache), Cache Friendly Collectioin
Thread Affinity and Isolation
Queue (LinkedList, Array) Ring Buffer, In Memory

Concurrency
Single Threaded, Fx 4.5 Async-Await
Journaler, Sequential Disk

Array is Cache Friendly.

64-bit Cache key, Concurrent Map/Segments, 1000 Segments. Producer never block to wait consumer

 Loop unrolling,Lock-coarsing, Inlining

Queue has fundamental Issues, Ring Buffer is better but on Dsktop cannot have 10M

Network: 10mcs local hop, 10GigE, FPGA market Data


http://mechanical-sympathy.blogspot.hk/

http://martinfowler.com/articles/lmax.html - even martin fowler has done a review of it

http://lmax-exchange.github.io/disruptor/ - all the source code is on github

http://www.infoq.com/presentations/LMAX - presentation on the architecture