Integration Issues

Unlike the situation with most programming languages, an Essence# programmer is expected to add to and even change namespaces, classes, traits and methods in the language’s standard library. And the same would be true of any namespacesclassestraits or methods defined (owned) by those who publish class libraries meant to be shared with and used by others (whether “open source” or otherwise.)

But that creates a problem whenever the time comes to acquire and integrate with a later version of a class library for which the programmer isn’t the principal author, but which he nevertheless extends or even modifies.

The Essence# Standard Class Library is one example where the problem of integration occurs. So let's examine the case of integrating with new releases of Essence# as a good example of the issues and/or problems raised by the need to integrate with new releases of "the same" class library, and explain the recommended solutions: 

Integrating with new releases of Essence#

The installer for each new release of Essence# will use the same default installation path, which means that, unless you choose a different installation path each time, or relocate the installation folder for the previous release each time, each installation of a new release of Essence# will be installed on top of the installation folder at the default installation location, and that folder will contain the files for a previous release, unless you take action to prevent it.

If you don't take action prevent the installation of a new release from overwriting a previous one, then any namespacesclasses or traits in the Standard Library--or in any other class libraries that are part of the standard Essence# installation--that were previously installed at the same location will have their methods and configuration definitions completely overwritten by the installation of a new release. You can prevent such overwriting by using a different installation path, or else by relocating the folder(s) containing the previous installation before installing the new release.

In contrast, any namespaces, classes or traits added locally, or removed as part of the new release, would not be affected by installing a new release on top of a previous release. However, the fact that any namespaces, classes or traits that have been obsoleted and removed from a new release won't be removed from the files residing in the selected installation location (when installing a new release on top of a previous release) may cause serious problems.

There are several ways to prevent such problems:

  • Choose a different installation path for each new release.
  • Create a copy of the files/folders created by the Essence# installer, and use those as your working environment. Of course, this approach requires manually changing the value of the %EssenceSharpPath% environment variable so that it points to your working copy of the Essence# standard library and other files, and also requires deleting or relocating the installation folder(s) of any previous release that already inhabits the location you choose when running the installation program for the new release, so this approach may be too burdensome.
  • Never change anything in the hierarchy of %EssenceSharpPath%\Source\Libraries\Standard.lib, and always delete or relocate the folder(s) that contain the previous release before installing the new one. It is important to note that this does not mean you can't change classestraits or namespaces in the Standard Library; it just means you can't do it by directly modifying the files and folders that reside in the hierarchy under %EssenceSharpPath%\Source\Libraries\Standard.lib.  There is another, better way to extend or modify the namespaces, classes or traits defined by the Essence# standard library. To see what that is, read on.

Managing Conflicts Between Class Libraries And/Or Projects, And Different Releases/Versions Thereof

To understand the recommended solution patterns, it is first necessary to understand how class libraries work in Essence#:

An Essence# class library is a container of namespaces, which in turn are containers of classestraits and/or other namespaces. Of course, that’s pretty much the definition of both of those terms in most modern programming languages, so there's nothing new there.

Here’s where Essence# diverges from the norm:

  • An Essence# class is also a namespace. It has all the same behavior, capabilities, features and functions as an Essence# namespace. And Essence# namespaces, classes and traits are objects. So they can be sent messages at run time. And those messages can modify or even totally redefine the namespace, class or trait to which they are sent.
  • It’s possible for the same class, trait or namespace to reside in multiple namespaces–even under different names–because the identity of a class, trait or namespace at runtime is a function of its identity as an object, and not a function of its name.
  • And it’s also possible for the same namespaces, classes and traits to be defined/declared in different class libraries. Two different Essence# class libraries may define completely different namespaces, classestraits and methods, or they may define all the same ones, or (far more likely) they may mostly define different namespaces, classestraits and methods, but have some (usually small) overlap. This is possible because namespaces, classestraits and methods are not created and grouped together declaratively, but instead by means of message sending. If namespaces, classes and methods were defined declaratively, then any differences between how they are declared in one class library and the way they are defined in another would create a logical contradiction, because logical declarations are universal invariants which must hold everywhere for all time. Since Essence# namespaces, classes and traits are intrinsically dynamic, differing definitions do not create a system panic: The system is designed to permit such things to change over time. So, if “the same” class can differ from itself from one moment to the next as the program runs, then it can certainly differ from itself with respect to how it is defined in various and sundry class libraries. Therefore, such differences need not–and do not–cause any errors at compile time.
  • An Essence# code base can and does contain what would, in a static language where classes are defined declaratively so that their name, form and behavior are invariants of the system, have to be treated as “logical contradictions,” and so disallowed as compilation errors. In Essence#, one dynamically specifies which class libraries will be loaded into the execution environment, and in what order. That is how different class libraries are merged together into a unified whole for a specific invocation of a program, where the last library loaded “wins” any disagreement over the definition of a namespace, class, trait or method. Even better, a class library that merely wants to extend a class by adding a method can do so without a) knowing all the methods defined for the class by any other class library, and b) without thereby deleting all those methods it knows nothing about. The same is true for any methods it modifies. To put it another way, Essence# does not operate using the “closed world assumption” that if a class library does not assert a fact about a class, then that fact must be false. The messages a class library sends that construct or modify a namespace, class or trait are never interpreted by the system as the final and complete definition of the namespace, class or trait, to the exclusion of the “declarations” made in other class libraries about the same class.
  • Finally, Essence# class libraries are always stored separately from each other. That means they can change independently of each other, even when they define, or modify the properties of, the same namespace, class, trait or method. That also means that that a new version of one library has no effect on the code of another, ever–even if they define the same namespace, class, trait or method. Yes, they still may conflict with respect to the form, structure or behavior of a namespace, class or trait, but such conflicts would only matter when both are loaded into the same execution context. More importantly, such conflicts can be resolved by yet another class library whose sole purpose and function is to modify the namespaces, classes, traits and methods that have been loaded into the execution environment so that they operate together usefully, and as intended by whoever put them together into a unified solution.

Key facts:

  • If two class libraries that are loaded into the same execution context declare "the same" namespace, "the same" class or "the same" trait, then that namespace, class or trait will only be created once. Any modifications to a namespace, class or trait made by any class library will affect that trait, class or namespace globally within the execution context of the running program. A namespace, class or trait defined in one class library is "the same" as one defined in another if their fully-qualified names are identical.
  • Therefore, if you create a class library named Standard.Local as a sibling of %EssenceSharpPath%\Source\Libraries\Standard.lib, then any classestraits or namespaces you define in your "Standard.Local" library that have the same fully-qualified names as any class, trait or namespace in the "Standard.lib" library will be declaring, defining, configuring and/or adding methods to the exact same classestraits or namespaces as are defined in the "Standard.lib" library. And since the library loader always compiles what's in the "Standard.lib" library first, any namespaces, classestraits or methods you specify in your "Standard.Local" library will modify any defined in the "Standard.lib" library. That means you can easily--and without fear of conflicts or collisions when installing a new release--add new instance variables, add new methods, or even change existing methods of any class or trait defined in the "Standard.lib" library just by specifying such changes in some other class library.
  • It should be noted that the methods declared for a class or trait in the "methods.instance" or in the "methods.class" files for a class or trait do not have to be duplicated in their entirety when redefining them in some other class library; you need only declare those namespaces, classestraits or methods you are adding, or re-declare those you are changing.

So Essence# handles integration conflicts by a) keeping the “(source or object) code of record” of class libraries distinct from each other permanently, and not just as a necessity of packaging and/or distribution, b) enabling full dynamic control over which class libraries are loaded into the execution environment created when a program is launched, and providing full control over the order in which the class libraries will be loaded, and c) using and embracing the dynamic malleability of objects, including those that are namespaces, classestraits or methods, instead of fighting against it.


In light of the above, it is recommended that the following guidelines be followed:

  • Use different values of the %EssenceSharpPath% environment variable for binding to different releases/versions, different applications, different projects or different frameworks (or any combination thereof.)
  • Use different class libraries (perhaps with related names) to hold the code for different releases/versions, applications, projects or frameworks (e.g, NinjaTrader.7 vs. NinjaTrader.8, or Squeak.compat vs. VisualWorks.compat.)
  • Use different class library and/or script search paths.
  • Use different namespaces (preventing such conflicts is the primary motivation for having namespaces in the first place.)
  • Use different configuration profiles.
  • Use a source code control/versioning system.  The code that defines the initial state of an Essence# execution environment resides in the file system as source code (text,) so there's no reason not to do this.

Last edited Aug 9, 2014 at 3:19 AM by Strategesis, version 53