Recently, I have been picking up Python with renewed interest now that a lot of libraries I'm interested in have stable Python3 releases. Perhaps, again, it is conditioning, but I really love a good IDE when using a language; especially an IDE that gets out of my way, and allows me to be productive. For me, the biggest advantage of a good IDE is object/class/library browsing. For this example, I used Visual Studio 2015 Community and the Python Tools for Visual Studio to flawlessly setup my Flask project, configure my Virtual Environment, and manage my requirements.txt.

You can find the source code for this project on GitHub.

In my last post, I did some benchmarking of Go, Gin, and Heroku. I chose Heroku because it is fairly trivial to spin up a new Heroku app: just install the toolbelt, and inside your git repo, run:

$ heroku create
$ git push heroku master

Also, another interesting aspect to Heroku is that we are deploying our software to a shared PaaS environment on top of some sophisticated virtualization, containerization, and service orchestration layers. Your compute resources are shared, and this can seriously affect the performance characteristics of your software.

(I understand I'm not saying anything revolutionary about Heroku, but these comments are for readers that might be entirely new web development and using a Platform as a Service like Heroku or Azure.)

Keep doing one thing well

Again, I am a huge proponent of the first axiom of UNIX philosophy:

Make each program do one thing well. To do a new job, build afresh rather than complicate old programs by adding new "features".

Therefore, from a business perspective, we can roughly state that our Python3/Flask exammple fulfills the same business objectives as our previous Go/Gin example. The both produce a JSON object containing an "rng" and a random number generated by a Mersenne Twister PRNG.

As stated previously, the Mersenne Twister is often used in regulated gaming (casinos/Class III, Class II, etc.) as an acceptable software psuedo random number generator. Gaming has different P/RNG needs than, say, cryptography, so using a deterministic algorithm like the Mersenne Twister is perfectly acceptable so that labs can verify the randomness of the PRNG over so many iterations.

In our Go, Gin, and Heroku example, we used an implementation of the Mersenne Twister written by GitHub user @CasualSuperman. In Python 3's standard library, the Mersenne Twister is the default PRNG in random.random():

Almost all module functions depend on the basic function random(), which generates a random float uniformly in the semi-open range [0.0, 1.0). Python uses the Mersenne Twister as the core generator. It produces 53-bit precision floats and has a period of 2**19937-1. The underlying implementation in C is both fast and threadsafe. The Mersenne Twister is one of the most extensively tested random number generators in existence. However, being completely deterministic, it is not suitable for all purposes, and is completely unsuitable for cryptographic purposes.

Interpreted vs. Compiled

The standard, default implementation of Python, CPython, provides a relatively fast execution environment for our Python code. Since the underlying Mersenne Twister in Lib\random.py is written in C (as noted above), I was certain that we would see some impressive results for an interpreted language. Python did not disappoint. In the future, I'd like to try Jython due to the amazing optimization characteristics of long-running software on top of the JVM.

Go, on the other hand, is interesting due to its garbage collection in addition to being compiled to machine code.

So, while from a technical perspective, comparing an interpreted Python application to a compiled Go application may seem unfair, I tend to look at tooling comparisons, at a high level, from a business and operations perspective.

At the end of the day, I really like creating durable software that doesn't require a high level of operational maintenance and babysitting. As noted above, from a business perspective, the Python3/Flask and Go/Gin demo micro services fulfill the same business function.

So, the question becomes, is Flask better than Gin at reducing operating expenses (OpEx) due to reduced compute resources needed to serve our application at scale? And, do those OpEx savings come with hidden costs in engineering time?

Some code

Here is the small snippet that handles the default route serving up our JSON:

from flask import render_template, jsonify  
from demoservice_flask import app  
import random

@app.route('/')
def generate_rng():  
    """Renders a random number using Python's built-in Mersenne Twister."""
    return jsonify(rng = random.random())

Meat and Potatoes

We ran the same 5 test conditions, using Loader.io, as we did for our Go/Gin test.

Test 1

Test 2

Test 3

Test 4

Test 5

Not even close

Not surpisingly, the Go/Gin performance (over 300,000 without a single HTTP 40x/50x) completely dwarfed the best Python3/Flask response of 49,322 responses in 1 min.

These tests could be more accurate, but they serve as a high level exploratory test of performance capabilities of a single task. Load testing a web service requires testing every interaction between end-points in the system.

Should I use Flask or Gin?

Like most aspects of software development and engineering, the answer is:

It depends.

Performance isn't the only useful metric by which to decide your tools. Computations time is usually far less expensive than developers' time.

There are pros and cons to each languae and framework. Compared to Python, Go is rather young, and while there are many libraries, answers on StackOverflow, and blogs, the simple fact of the matter is that, today, Python has a much larger community.

And community matters when scouting for talent.

However, it may be that Go provides you performance characteristics that provide a huge benefit to your problem domain, or provides a competitive advantage against your competitors; especially at scale.

Then again, if a team can't get features to market fast enough, it doesn't really matter how fast their code runs.

The real problem is that programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming.

Donald Knuth, "Computer Programming as an Art (1974)"

The performance characteristics of Go may be a premature optimization for your project. Then again, maybe not.

It depends.

I'm not sure if it is conditioning, or aesthetics, but I enjoy C-style syntax, and I feel at home in languages that follow similar semantics. As a full stack web developer, I am constantly looking for new techniques to build services and applications at scale. Surprisingly enough, I'm just now taking a look at Google's Go.

Having recently completed the design and implementation of an online game system to support lottery games built in Unity 3D (and targeting WebGL), I am interested in using hindsight to assess how I might have engineered various pieces a bit differently. In the regulated gaming industry, the Mersenne Twister is often used as a software pseudo-random number generator (PRNG). Since RNGs form a crucial component of various regulated gaming systems, it seems creating a durable RNG service that combines speed and stability is a high priority.

(In the future, we may explore porting this example to Erlang, Elixir, and other slightly esoteric languages.)

Do one thing well

As a huge fan of "The UNIX Philosophy," it always occurs to me that a micro-service should do one task, and do it rather well.

The code for this demo service is on GitHub and released under the MIT License.

Here is the main code listing (with some thanks to @CasualSuperman on GitHub for the Mersenne Twister in Go):

The code

package main

import (  
    "log"
    "net/http"
    "os"
    "sync"
    "math/rand"
    "github.com/gin-gonic/gin"
)

// https://github.com/CasualSuperman/Mersenne-Twister
type Generator struct{  
    state [624]uint32
    index uint32
    generating sync.Mutex
}

func Init(a uint32) *Generator{  
    var t Generator
    t.init(a)
    return &t
}

func (g *Generator) init(a uint32){  
    g.generating.Lock()
    g.state[0] = a
    var i uint32 = 0
    for i=1; i<624; i++{
        g.state[i] = uint32(0x6c078965 * (g.state[i-1] ^
        g.state[i-1]>>30) + i)
    }
    g.generating.Unlock()
}

func (g *Generator) Next() uint32{  
    g.generating.Lock()
    if g.index == 0{
        g.generateNumbers()
    }
    y := g.state[g.index]
    g.index++
    g.index %= 624
    g.generating.Unlock()
    y ^= y>>11
    y ^= y<<7 & 0x9d2c5680
    y ^= y<<15 & 0xefc60000
    y ^= y>>18
    return y
}

func (g *Generator) generateNumbers(){  
    var i uint32
    for i=0; i<624; i++{
        y := g.state[i]>>31 + g.state[(i+1)%624] & 0x7fffffff
        g.state[i] = g.state[(i+397)%624] ^ y>>1
        if y%2 == 1{
            g.state[i] ^= 0x9908b0df
        }
    }
}

func main() {  
    port := os.Getenv("PORT")

    rng := Init(rand.Uint32())

    if port == "" {
        log.Println("$PORT must be set.  Setting to 8080.")
        port = "8080"
    }

    router := gin.New()
    router.Use(gin.Logger())
    router.Static("/static", "./static")

    router.GET("/loaderio-fad37ceadee65cb4bdf707027748bd42", func(c *gin.Context) {
        c.Writer.Write([]byte("loaderio-fad37ceadee65cb4bdf707027748bd42")) // Passes Loader.io verification :)
    })

    router.GET("/", func(c *gin.Context) {
        //c.HTML(http.StatusOK, "index.tmpl.html", nil)
        c.JSON(http.StatusOK, gin.H{
            "rng": rng.Next(),
        })
    })

    router.Run(":" + port)
}

Dyno Might

For this project, I decided to deploy our Go micro service, written using the Gin framework, to a free Heroku instance. To be honest, I wasn't very surprised at the results (well, after a bit of trouble getting the slug to compile and bind to $PORT).

My reasoning is that squeezing some great performance out of a single free-tier Heroku dyno is an interesting exercise in tasting the potential of Go micro services. Also, I didn't want to provision my own dedicated application server, nor add this micro service to an existing application server.

(In the future, it wil be interesting to see if I can get this micro service running on an Azure Service Fabric cluster; both on localhost, and on Azure.)

Measure twice, then cut

Here are some results from Loader.io. Here, I also used their free plan; particularly since their free plan is adequate to test my single end-point.

Test 1

Test 2

Test 3

Test 4

Test 5

Over 300,000

Test 5 was the most successful. In one minute, we were able to serve up 374,307 pseudo-random numbers with each request returning a 200 OK.

374,307 successful requests/responses. Not a single 400/500 error. On a single free dyno.

And that with an average response time of 190 ms.

Delicious. ;)

FFP is now accepting manuscript submission for all types of poetry and prose; everything from novels, to art books, to compilations, to tabletop role playing game supplements and core systems.

FFP is a full-service small press, and we are looking forward to serving authors and audiences of all persuasions.

After all, Full Flavor Press firmly believes that freedom of speech is more important than political correctness.

Let's write and publish together.

ERP systems and packages (like SAP, Epicor, Dynamics AX/NAV/GP, Oracle ERP, etc.) have their roots in automating manufacturing processes in the 1960's and 1970's. A quick glance at database schemas, heavy reliance on procedural programming, licensing models, and user interfaces of these diverse ERP systems shows their age.

Terrible Architecture & Closed Source

From a software architecture perspective, one of the largest problems with commercial ERP systems is that they try to be all things to all businesses while shoehorning an organization's business processes into the opinionated workflow of the ERP vendor. Any deviations from the vendors' conflicting visions comes at great cost to an organzition; both in operating expenses, and capital expenses - not to mention the opportunity cost of implementing ERP at the expense of internal innovation. Add, on top of this, integration with existing business systems, and the costs become astronomical.

Not only are the software architectures of existing ERP systems too mature for their own good (meaning, beneficial changes are slow to come and hard to integrate into brittle systems), the source code for the typical ERP system is closed. This means orgnaizations lose the ability to draw on the software development expertise of thousands of developers at no additional cost to the organization implementing an ERP system.

High-Level Cost Estimates for Dynamics AX

Let's look at three common deployment scenarios for Dyanmics AX:

• 3 concurrent users, Financial Modules Only (~$20,000 - $30,000 USD):
  • Software: ~$5,000 - 8,000.
  • Implementation Services: ~90 to 120 hours of consulting fees including brief requirements analysis (~8 hours), setup, configuration, testing, and training.
  • Hardware Purchase and Deployment: ~$12,000 - $20,000 USD if new hardware hasn't been purchased (not including hypervisor and operating system licensing fees).
• 7 - 10 concurrent users, Financials, Sales, & Inventory Modules (~$60,000 - $90,000 USD):
  • Software: ~$30,000 USD.
  • Implementation Services: ~220 to 320 hours of consulting fees including requirements analysis, setup, configuration, testing, training, and go-live support.
  • Hardware Purchase and Deployment: ~$25,000 - $40,000 USD if new hardware hasn't already been purchased (not including hypervisor and operating system licensing fees).
• 12 - 15 concurrent users, Financials, Sales, & Inventory Modules (~$120,000 - $180,000 USD):
  • Software: ~$48,000 USD.
  • Implementation Services: ~300 to 450 hours of consulting fees including requirements analysis, setup, configuration, testing, training, and go-live support.
  • Hardware Purchase and Deployment: ~$35,000 - $50,000 USD if new hardware hasn't already been purchased (not including hypervisor and operating system licensing fees).

None of these scenarios include support beyond the first year within the purchase date of the software (support usually starts as soon as you buy, not when you deploy). It should also be noted that the above scenarios assume the highly unlikely scenario that the implementation will happen on time, within budget, and not need any customizations for an organization's existing business processes.

A Better Way

ERP systems don't have to be rigid, monolithic, wastes of capital. By using modern, well-architected ERP software supported by an open source community, flexibility and adaptability become first-class citizens in enterprise.

Arcana ERP is a modern, open source, enterprise resource planning system built upon the OpenCLR and .NET stacks (allowing deployment to Windows Server and Linux infrastructure) supporting a wide range of database back-ends; from SQL Server, to Oracle, to MySQL, to PostgreSQL.

Arcana ERP is built for highly concurent and distributed work-loads using funtional programming and software transactional memory built upon a micro-services architecture built for fault-tolerance and performance. (Other blog posts go father into depth on the architecture and development of Arcana ERP).

Open Source Software Success Stories

The success of open source software can be be seen with:

• Linux
• Apache Software Foundation
• GNU
• Java
• Ruby on Rails
• Python
• R
• MySQL
• PostgreSQL
• Git
• SVN
• OpenCLR
• Too many more applications to count in a single blog post!

Stop Wasting Hard-Earned Capital

Consider trying Arcana ERP today. Arcana ERP has feature parity with large, entrneched packages like SAP and Dynamics AX, but with full-featured and easy to install open source packages (with well documented APIs for your Developers :). You can do a complete in-house installation and deployment of Arcana ERP without ever paying a single license fee for Arcana ERP; meaning, your organization can eliminate exorbitant annual licensing fees while drastically reducing development and support costs.

Commercial Support and Custom Development

Although Arcana ERP gives you broad and deep functionality our of the box, we understand the need for commercial support and Service Level Agreements. The Wizard & The Wyrd, LLC offers Arcna ERP long-term support and custom development at a fixed-rate of 25% of your current annual ERP licensing fees. We will never raise our rates, and you are not required to share any custom developments with the rest of the open source community (we understand the need to protect intellecutal property and the competitive advantages of your well-tuned business processes).

An interesting outcome of this experiment is that we can see concise, expressive, and terse idiomatic F# "expand" into equivalent, and more verbose, C#:

F# Source

namespace ArcanaErp.Core.Lambda

open System

module Interfaces =

    type IBaseErpModel =
        abstract member Id : int with get, set
        abstract member Description : string with get, set
        abstract member ExternalIdentifier : string with get, set
        abstract member ExternalIdSource : string with get, set
        abstract member CreatedAt : DateTime with get, set
        abstract member UpdatedAt : DateTime with get, set

dotPeek'd C#

using Microsoft.FSharp.Core;  
using System;

namespace ArcanaErp.Core.Lambda  
{
  [CompilationMapping(SourceConstructFlags.Module)]
  public static class Interfaces
  {
    [CompilationMapping(SourceConstructFlags.ObjectType)]
    [Serializable]
    public interface IBaseErpModel
    {
      int Id { get; set; }

      string Description { get; set; }

      string ExternalIdentifier { get; set; }

      string ExternalIdSource { get; set; }

      DateTime CreatedAt { get; set; }

      DateTime UpdatedAt { get; set; }
    }
  }
}

A Bigger Example

First, in F#

The previous example was fairly straightforward. We see some extra curly braces and some attributes we can get rid of. Here is a sample from the FSharp.Core assembly in the Microsoft.FSharp.Collections namespace:

        [<CompiledName("IterateIndexed")>]
        let iteri f (source : seq<'T>) = 
            checkNonNull "source" source
            use e = source.GetEnumerator()
            let f = OptimizedClosures.FSharpFunc<_,_,_>.Adapt(f)
            let mutable i = 0 
            while e.MoveNext() do
                f.Invoke(i, e.Current);
                i <- i + 1;

Decompiled to C#

    [CompilationArgumentCounts(new int[] {1, 1})]
    [CompilationSourceName("iteri")]
    public static void IterateIndexed<T>(FSharpFunc<int, FSharpFunc<T, Unit>> action, IEnumerable<T> source)
    {
      if ((object) source == null)
        throw new ArgumentNullException("source");
      System.Collections.Generic.IEnumerator<T> enumerator = source.GetEnumerator();
      try
      {
        OptimizedClosures.FSharpFunc<int, T, Unit> fsharpFunc = OptimizedClosures.FSharpFunc<int, T, Unit>.Adapt(action);
        int num = 0;
        while (enumerator.MoveNext())
        {
          fsharpFunc.Invoke(num, enumerator.Current);
          ++num;
        }
      }
      finally
      {
        IDisposable disposable = enumerator as IDisposable;
        if (disposable != null)
          disposable.Dispose();
      }
    }

Wrapping Up

It is clear to see that F# is the winner here. The F# code is more compact, more expressive, and easier to reason about (after, admittedly, a small learning curve on the F# syntax).

Writing software as complex as an ERP systems, and enterprise software in general, requires code that can be reasoned about fairly easily. On all accounts, for us, F# makes sense.

The Ruby language, compared to other languages used to build and deliver web applications, has a comparitively small list of disclosed exploits on the CVE (Common Vulnerabilities and Exposures) list.

For reference, here is a list of historical CVE disclosures to date for a handful of languages/platforms used to develop web applications:

1. Python: 23
2. Ruby: 45
3. PHP: 409
4. Oracle JRE: 413

Here is the result of running Flawfinder 1.31, with the default rules in the Homebrew distribution, against the master branch of the current Ruby 2.2.x source code:

ANALYSIS SUMMARY:

Hits = 636  
Lines analyzed = 197285 in approximately 3.93 seconds (50139 lines/second)  
Physical Source Lines of Code (SLOC) = 141537  
Hits@level = [0]   0 [1] 246 [2] 288 [3]  35 [4]  52 [5]  15  
Hits@level+ = [0+] 636 [1+] 636 [2+] 390 [3+] 102 [4+]  67 [5+]  15  
Hits/KSLOC@level+ = [0+] 4.49352 [1+] 4.49352 [2+] 2.75546 [3+] 0.72066 [4+] 0.473374 [5+] 0.105979  
Minimum risk level = 1  
Not every hit is necessarily a security vulnerability.  
There may be other security vulnerabilities; review your code!  
See 'Secure Programming for Linux and Unix HOWTO'  
(http://www.dwheeler.com/secure-programs) for more information.

These results, at first glance, may prove interesting:

./addr2line.c:449:  [4] (buffer) strcpy:
  Does not check for buffer overflows when copying to destination (CWE-120).
  Consider using strcpy_s, strncpy, or strlcpy (warning, strncpy is easily
  misused).
./addr2line.c:450:  [4] (buffer) strcpy:
  Does not check for buffer overflows when copying to destination (CWE-120).
  Consider using strcpy_s, strncpy, or strlcpy (warning, strncpy is easily
  misused).
./addr2line.c:723:  [4] (buffer) strcpy:
  Does not check for buffer overflows when copying to destination (CWE-120).
  Consider using strcpy_s, strncpy, or strlcpy (warning, strncpy is easily
  misused).

For the complete test results, see this Paste.

Simply scanning the Ruby source of this particular implementation of Ruby with a single static analysis tool should be considered a preliminary exercise for more research.

The snow would thaw, the rivers begin to run and the world would wake into itself again.

So wrote Neil Gaiman in "Odd and The Frost Giants."

Today, in 2015, we see an encroachment of Ariticial Intelligence, under the form of deep machine learning, penetrating nearly every aspect of our interconnected social lives.

Unfortunately, like most advancements in the sum of human knowledge, this amazing technology is being used to siphon more dollars from your provate coffers. Afterall, the greatest virtue of modern business appears to be maximizing value for shareholders.

Let me be perfectly clear: as a small business, we love revenue and profit. What we don't love is the commoditization of the invasion of personal privacy to slake the lusts of marketing executives in their never-ending quest to sell you garbage. We are critical of Facebook's strategy to pursue deep machine learning in order to creat an even more effective advertising platform.

Of course, painting in broad strokes while creating generalizations is fraught with the danger that communication will degenerate into disinformation and propaganda. However, we are not convinced that the ubiquitous proliferation of social media over the past decade is an automatic net benefit to the prosperity of humanity.

Yes, information technology, and social media in particular, are allowing groups of dissatisfied populations the ability to coordinate everything from revolutions to corporate boycotts.

But at what price?

I think most of us, at some time or another, have felt an overhwelming feelings of desire to achieve personal excellence along a particular path; whether that be something as abstract as "entrepreneurship" or some definite goal like "published author" or "professional game developer."

But is passion really necessary for any measure of success?

I often find that consistent self-discipline is far more useful to success than any amount of passion. A long view of history means we should take the perspective that we take the immortal perspective of having omnipotence and eternity at our disposal while being acutely aware that time is a finite resource.

We can always find new lovers, earn more money, have new adventures, play more games, and drink more beer. However, every passing day rapidly depletes our store of Time which we find rapidly appreciates in combinatorial explosions of value as supplies accelerate towards oblivion in lock-step with our mortality.

If popular notions of reincarnation are somehow objectively true, then there is no reason to delay doing whatever needs to be done - today! - to come closer the far-end of the bell curve of human potential so that we can reap the rewards of our creative success for lifetimes to come (even if that success comes posthumously!).

If popular notions of reincarnation are nothing more than confortable lies we tell ourselves to soothe the knowledge that every breath hurls us closer to the grave, there is no reason to avoid doing whatever needs to be done - right now! - to traverse the abyss of time, pregnant with habitually discipline, that separates the average from the legendary.

Who are we?

The Wizard & The Wyrd, LLC, a small studio with big aspirations, was formed by long-time friends Ramon J. Long III and Jeremy Dolan. Jeremy is a sleeping titan waiting to unleash his illustrative talents upon a global audience, and Ramon is a writer with a gruesomely psychedelic and mystical story inspired by the underbelly power elite.

What is The Wizard?

The Wizard represents the primordial archetype of the cunning folk, of the wise man versed in the mysteries of cosmos (including its numerous heavens and hells). The Wizard is the master of Magick whereby, through his art and science, his Will is made to reverberate throughout the Universe is a wide variety of forces and forms.

What is The Wyrd?

Wyrd is a term for concepts roughly corresponding to those of fate or destiny but involving complex interactions of universal necessity and individual choice within a cosmos beyond any fixed notions or concepts of mortal minds.

Since the Autumn of 2002, Ramon has been brooding over creating an independent studio to publish the works of his extremely talented friends, cohorts, and accomplices. A lot has changed since then, and this vision to create a multi-media production studio - on our own terms - remains unchanging, and we have begun working on our first comic series; a six issue comic series that will be bound into a hard-cover graphic novel.