TFS Branching and Merging draft Guideline

Branch and Merge is a core concept in TFS to facilate Source Control and release
 management. There are various approach to these topics. But here are brain-dump
 while encountering practical issues using TFS ( Team Fundation server).

(1) Agile Development Directory Structure representing Team, products, solutions,
 major branch

      Trading System
            CRD
            XIP
              GMO.XIP
                 Branches
                   2008
                   2009
                 Trunk


(2) Each branch represent a weekly release version. and Trunk should not be released.
    This will allows us to support different usage of XIP solution for different 
    business cases and still support common core.
    If CMMI are used, then the structure could be based on version number not related to Dates

(3) For non-release related feature (eg. new helper function or bug fixes), a 
feature branches should be used for the changes. In other words, Trunk can only be
changed through merge back from branches.

(4) Branching should be off Tip Revision (Latest Version), rather than a label since TFS needs 
to duplicated files on branching and we have to use duplicated directory structure, which in
 itself already acts like Label.

Use reflection to dump properies

 For debugging purposes, we can use the following reflection based code:

       public static  string  DumpOrderBean(OrderBean ob)
        {
            
            PropertyInfo[] pi = typeof(OrderBean).GetProperties();
            StringBuilder sb= new StringBuilder();
            foreach (PropertyInfo p in pi)
            {
                if (p.Name.IndexOf("Bean")<0)
                sb.Append(p.Name +" :"+ p.GetValue(ob, null)+"\r\n");
            }
            StreamWriter w= new StreamWriter(@"c:\JGBRollTestOB.txt");
            w.Write(sb.ToString()); w.Flush();

            OrderAllocationBean oa = ob.allocationBeans[0];
            PropertyInfo[] pi2 = typeof(OrderAllocationBean).GetProperties();
            StringBuilder sb2 = new StringBuilder();
            foreach (PropertyInfo p2 in pi2)
            {
                if (p2.Name.IndexOf("Bean") < 0)
                sb2.Append(p2.Name + ":" + p2.GetValue(oa, null) + "\r\n");
            }
            StreamWriter w2 = new StreamWriter(@"c:\JGBRollTestOAB.txt");
            w2.Write(sb2.ToString()); w2.Flush();
            return sb.ToString();
        }

Constrain SQL like pattern for Commission Rate lookup

 SQL Like allow pattern match but normally return multiple rows. If you have one 
commision rate for a pattern, carefully specified pattern will allow look up to 
return just one value without the ambiguity as shown here:
  FSMI% vs. FSMIU%

In general, we need to consider the following:

(1) Clearly define Bloomberg symbol as 2 charaters key and Add GMO prefix to 
generate external security id pattern. (GMO_EXT_SEC_ID_PATN_DEF).
    In particular, Commission Rate table GMO_BROKER_COMM need to have Foreign key 
constraints from GMO_EXT_SEC_ID_PATN_DEF to avoid invalid security.

(2) GMO_BROKER_COMM need to have a trigger constraint to make sure the following
 will not return more than one row:
   select * from GMO_BROKER_COMM2 where 
     (EXT_SEC_ID_PATN like @EXT_SEC_ID_PATN or
           @EXT_SEC_ID_PATN like EXT_SEC_ID_PATN)

in particular, if we need to have both FSMI% and FSMIU%, we will need to span out
even with duplcation rates:
       FSMIH%   2.95
       FSMIG%   2.95
       FSMIZ%   2.95

       FSMIU%   4.95
 
This will eliminate ambiguity.

(3) We need [BB_SYMBOL] [char](2) NOT NULL to keep space for Bloomberg symbol
    and that will avoid some ambiguity like FBC % vs. FBCC%

(4) FBC % vs. FBC%
    FB C  -- SET50 FUTURES  maps to FBC %
    F  BC -- CORN FUTURE    maps to FBC% 
    we currenlty do not have commission rate for FBC %. But we need to be 
    careful when enter rates in the future

(5) The following is the SQL used in Mid-Tier:
    select * from GMO_BROKER_COMM where @ext_sec_id like EXT_SEC_ID_PATN

Runing CRD 8.2.2 against SQL Server 2005 Express

There are some particular considerations when install CRD 8.2.2 against SQL Server 2005 Express:

(1) SQL Server 2005 Express must installed as default instance. No other instance should ever be installed.
(2) Installation works on XP SP2.
(3) Must create TM_DEV user in login first before restore data.bak. And server role should be empty, especially should not have "public" role visible.
(4) Must use SQL Configuration manager to enable Client VIA, TCP/IP, ShareMemeory, NamedPipes.
(5) Create a new Database CRD822 and install into it with "Overwrite Existing DB" option checked.
(6) TM_DEV mapping to PDF_User dbo_owner roles. ( start up server while doing CRD client installation).

LINQ to Text file through StreamReader Extension method

Text file data feeds come in different format but with common theme: parse a line into an object representation. Here is one example of the object:

   public class BrokerDataItem : ILineParser<BrokerDataItem>
    {
        public string Ticker { get; set;}
        public double Price { get; set;}
        public long QTY { get; set;}
        public string Trader { get; set;}
        public DateTime TradeDate { get; set;}
        public string TradeType { get; set;}

        public BrokerDataItem() { }
         public BrokerDataItem Parse(string[] p)
        {
            return null;
        }
        public BrokerDataItem Parse(string entireLine) {
            string[] p = entireLine.Split(',');
            Ticker = p[0].ToString();
            Price = Convert.ToDouble(p[1]);
            QTY = Convert.ToInt64(p[2]);
            Trader = p[3].ToString();
            TradeDate = Convert.ToDateTime(p[4]);
            TradeType = p[5].ToString();
            return this;
        }

Note I abstracted parse operation into interface 

    public interface ILineParser<T>
    {
        T Parse(string[] p);
        T Parse(string entireLine);
    }

So that I can create an Extension Method for StreamReader that return the Data Object to LINQ. Consequently, this allows LINQ runs against data object of any source to get the required data elements:

    public static class AnyThing
    {
        public static IEnumerable<T> GetData<T>(this StreamReader sr) where T : ILineParser<T>, new()
        {
            string L;
            while ((L = sr.ReadLine()) != null)
            {
                T t = new T();
                yield return t.Parse(L);
            }
        }
    }


    class Program
    {
        static void Main(string[] args)
        {
            StreamReader sr = new StreamReader(@"..\..\Data.txt");

            var t =
                from data in sr.GetData<BrokerDataItem>()   
                select data;
            foreach (BrokerDataItem i in t)
            Console.WriteLine(i.Ticker+" " + i.Price.ToString());
            Console.ReadLine();
        }
    }

Here are the data:
FBN U8,134.79, 1000,JIMMYD,7/14/2008,BUYL
FBN U8,444.90, 7832,JIMMYD,7/14/2008,BUYL
FBN U8,4567.9, 1000,JIMMYD,7/14/2008,SELL
FBN U8,581.79, 5634,JIMMYD,7/14/2008,BUYL
FBN U8,724.78, 1212,JIMMYD,7/14/2008,SELL
FBN U8,932.01, 8892,JIMMYD,7/14/2008,SELL
FBN U8,678.88, 3,JIMMYD,7/14/2008,BUYL

Finally, extension method is not required since you can just create an IEnumerable and use it in LINQ.

Data Proection API

DPAPI uses the key from the following location to encrypt/decrypt:

C:\Documents and Settings\All Users\Application Data\Microsoft\Crypto
or

C:\WINDOWS\system32\Microsoft\Protect or Crypto.

These are replaced by SID if DataProtectionScope.LocalMachine is replaced by DataProtectionScope.CurrentUser.
In theory, if data are moved away from the machine, there would no way to decrypt,
even DataProtectionScope.CurrentUser for the same user logged on to a different machine:

        byte[] entropyBytes =null;// Encoding.Unicode.GetBytes("GMO");
        byte[] pBytes;
        private void button1_Click(object sender, EventArgs e)
        {
            string plainText = this.textBox1.Text;
            byte[] plainBytes = Encoding.Unicode.GetBytes(plainText);

            pBytes = ProtectedData.Protect(plainBytes, entropyBytes, DataProtectionScope.LocalMachine);
           string ps = Convert.ToBase64String(pBytes);
           this.textBox2.Text = ps;

        }

        private void button2_Click(object sender, EventArgs e)
        {
            byte[] upBytes = ProtectedData.Unprotect(pBytes, entropyBytes, DataProtectionScope.LocalMachine);
            this.textBox3.Text = Encoding.Unicode.GetString(upBytes);
        }

Using WSE 3.0 X509 Search API to implement RSA Crypto

WSE 3.0 has two functions to retrieve X509 Certificate:

X509Certificate2Collection cert2s = X509Util.FindCertificateBySubjectName(SubjectName, StoreLocation, StoreName.ToString());

X509Certificate2Collection cert2s = X509Util.FindCertificateByKeyIdentifier(ThumbPrint , StoreLocation, StoreName.ToString());

This is much simpler than using COM API or its .Net Wrapper. Using these function, we can write a class encapsulate RSA Crypto with considering of Certification Basic Policy Validation ( such as revocation):


namespace JQD {
public class X509RSAEncryptor
{
  #region Encryption and Decryption
      public static string[] EncryptUTF8ToBase64(string ClearTextUTF8, X509Certificate2 cert2ForEncryption, X509Certificate2 cert2ForSignning)
    {
       if (null == cert2ForEncryption) 
           throw new ApplicationException("null X509 cert for Encryption");
      // create Encryption RSA using Public Key only
      RSAParameters rsaForEncryptionPublicParam = X509Util.GetKey(cert2 ForEncryption).ExportParameters(false);
      RSACryptoServiceProvider rsaForEncryption = new RSACryptoServiceProvider();
rsaForEncryption.ImportParameters(rsaForEncryptionPublicParam);
      // generate encrypted Base64 string from clear Text
      byte[] clearBytes = Encoding.UTF8.GetBytes(ClearTextUTF8);
      byte[] cipherBytes = rsaForEncryption.Encrypt(clearBytes, true);
      string cipherTextBase64 = Convert.ToBase64String(cipherBytes);
      string signatureTextBase64 = "";
     if (null != cert2ForSignning)
     {
      // create Signning RSA using Private Key
      RSAParameters rsaForSignningPrivateParam = X509Util.GetKey(cert2ForSignning).ExportParameters(true);
      RSACryptoServiceProvider rsaForSignning = new RSACryptoServiceProvider();
rsaForSignning.ImportParameters(rsaForSignningPrivateParam);
      // compute Signature using SHA1
      byte[] signature = rsaForSignning.SignData(cipherBytes, SHA1.Create());
signatureTextBase64 = Convert.ToBase64String(signature);
      rsaForSignning.Clear();
     }

      rsaForEncryption.Clear();
      // send both Data and its singature as Base64 string
      return new string[2] { cipherTextBase64, signatureTextBase64 };
   }
    public static string DecryptBase64ToUTF8(string[] CipherBase64, X509Certificate2 cert2ForDecryption, X509Certificate2 cert2ForVerifySignning)
    {
    if (null == cert2ForDecryption) 
       throw new ApplicationException("null X509 cert for decryption");
    byte[] cipherBytes = Convert.FromBase64String(CipherBase64[0]);
    if (null != cert2ForVerifySignning)
    {
    // create Verify Signning RSA using public Key
    RSAParameters rsaForVerifySignningPrivateParam = X509Util.GetKey(cert2ForVerifySignning).ExportParameters(false);
    RSACryptoServiceProvider rsaForVerifySignning = new RSACryptoServiceProvider();
rsaForVerifySignning.ImportParameters(rsaForVerifySignningPrivateParam);
    // Verify signature
    byte[] signature = Convert.FromBase64String(CipherBase64[1]);
   try
    {
   if (!rsaForVerifySignning.VerifyData(cipherBytes, SHA1.Create(), signature))  
      throw new ApplicationException("Data have been tampered.");
    }
    finally
    {
       rsaForVerifySignning.Clear();
    }
  }
  // create Decryption RSA using Private Key
    RSAParameters rsaForDecryptionPrivateParam = X509Util.GetKey(cert2ForDecryption).ExportParameters(true);
    RSACryptoServiceProvider rsaForDecryption = new RSACryptoServiceProvider();
rsaForDecryption.ImportParameters(rsaForDecryptionPrivateParam);
    // Decrypt and convert to Clear Text
    byte[] clearBytes = rsaForDecryption.Decrypt(cipherBytes, true);
    rsaForDecryption.Clear();
    return Encoding.UTF8.GetString(clearBytes);
  }
 #endregion

  #region X509 finder
   public static X509Certificate2 FindX509Certificate2(string SubjectName, StoreLocation StoreLocation, StoreName StoreName)
   {
    X509Certificate2Collection cert2s = X509Util.FindCertificateBySubjectName(SubjectName, StoreLocation, StoreName.ToString());
    X509Certificate2Collection validCert2s = Validate(cert2s);
    if (validCert2s.Count == 0) return null;
    return validCert2s[0];
   }
   public static X509Certificate2 FindX509Certificate2(byte[] ThumbPrint,StoreLocation StoreLocation, StoreName StoreName)
   {
   X509Certificate2Collection cert2s = X509Util.FindCertificateByKeyIdentifier   (ThumbPrint , StoreLocation, StoreName.ToString());
    X509Certificate2Collection validCert2s=Validate(cert2s);
    if (validCert2s.Count == 0) return null;
   return validCert2s[0];
   }
   private static X509Certificate2Collection Validate(X509Certificate2Collection cert2s)
   {
    X509Certificate2Collection validCert2s = new X509Certificate2Collection();
    foreach (X509Certificate2 cert2 in cert2s)
    {
    // applies the base policy to that chain, Note that on Win2k3, the basic policy
    // check conformance to RFC3280, which include revocation for X509 Cert2.
     bool IsConformedToBasicPolicy = cert2.Verify();
    // Some other simple checking
    bool IsArchived = cert2.Archived;
    bool IsExpired = (DateTime.Now > cert2.NotAfter)
    bool IsNotActive =( DateTime.Now < cert2.NotBefore);
    if (IsConformedToBasicPolicy && !IsArchived && !IsExpired && !IsNotActive)
    {
    validCert2s.Add(cert2);
    }
   }
    return validCert2s;
   }
 #endregion
 }
}

The usage is also simple as illustrated by the following:

class Program
{
      static void Main(string[] args)
      {
        string msg="This is a secret.";
        Console.WriteLine(msg);
        X509Certificate2 cert2Enc;
        X509Certificate2 cert2Sig;
        string subjectName1="CN=idmcertid, OU=Internet Infrastructure, O=\"Blah Blah, Inc.\", L=Boston, S=massachusetts, C=US";
        byte[] thumbPrint1=new byte[] { 0xb7, 0x4a, ....., 0x9c, 0x0c };

        cert2Enc = X509RSAEncryptor.FindX509Certificate2(subjectName1,StoreLocation.LocalMachine, StoreName.My);
        cert2Sig = X509RSAEncryptor.FindX509Certificate2("CN=XYZ Company",          StoreLocation.CurrentUser, StoreName.My);
        cert2Sig = X509RSAEncryptor.FindX509Certificate2(subjectName1, StoreLocation.CurrentUser, StoreName.My);
        string[] cipherBase64 = X509RSAEncryptor.EncryptUTF8ToBase64(msg, cert2Enc, cert2Sig);
       Console.WriteLine(cipherBase64[0]);
       Console.WriteLine();
       Console.WriteLine(cipherBase64[1]);
       Console.WriteLine();
       X509Certificate2 cert2Dec=X509RSAEncryptor.FindX509Certificate2(subjectName1,StoreLocation.LocalMachine, StoreName.My);
       Console.WriteLine(X509RSAEncryptor.DecryptBase64ToUTF8(cipherBase64, cert2Dec, cert2Sig));
       Console.ReadLine();
}

Note that when install Private Key, Must check " Mark this key Exportable" to avoid " Key in invalid state" exception. Also remember RSA encryption requires Receiving party to hold private key for decryption and hold public key for verify signning. Sending party hold just the opposite. So Receiveing party only need sending party's public key, and so on. This means it is better to have two computers with two set of good X509 Cert to run the code. If the cert is really invalid as seen in MMC IDE, IsConformedToBasicPolicy will be false.
Finally, There are no need to install WSE 3.0. You only need to have a copy of Microsoft.Web.Services3.dll and add its reference to your project.