You are currently browsing the category archive for the ‘Uncategorized’ category.

Update: See this Docker image.

People sometimes ask about making GemStone available in a Docker container. The demand seems to be driven by a desire for a simplified install/deployment process. In preparation for my ESUG talk on a cloud-hosted GemStone IDE I decided to do some investigation.

Docker is an alternative to a virtual machine (which I’ve written about extensively on this blog) where the guest environment (“container”) shares not just the hardware with the host and other guests, but also the OS kernel. What is isolated is the application and supporting libraries. It does allow simplified deployment and ensures that each installation has exactly the same libraries and other components. The container can also be isolated so that (by default) it does not write outside its boundaries. These are attractive features.


A typical description of Docker suggests that each container uses the host OS, but it really uses the host OS kernel, and on macOS it runs an embedded Linux instance. Thus, a single Docker container can run on various OS hosts. Contrary to my initial misconception, you don’t need a separate Docker container for every OS and version.

A more challenging issue is what should go in the container. Obviously, it should include GemStone (and you would need a different container for each GemStone version), but what else? Should it include a web server? If so, which one? Perhaps not, but it does increase the complexity if you need to install, configure, and coordinate multiple components.

One hurdle is that once built, the software in a container is essentially fixed. You do not upgrade the contained software, you replace the container. Furthermore, I hope your container does not have any persistent data – it’s not supposed to, meaning thou shalt not run a database inside a container. Or, at least, not a production database; running a development database inside a container might be a good idea.

More specifically, you need to make sure that any persistent data is held outside the container, meaning that your container is not so isolated. Volumes are the preferred mechanism for persisting data generated by and used by Docker containers. And you would still need to manage backups and related system administration tasks.

For things like Topaz (a command-line interface to GemStone), you could run a command in the container. For RPC Gems (used by most client applications and IDE tools such as Jade), you should need to have only one port open into the container since the server-side Gem would be inside the container as well.

Overall, I don’t find it very difficult to install GemStone, but if someone were more likely to investigate GemStone if a Docker container were available, then it shouldn’t be too difficult to provide it.


A question on the GLASS mailing list raised some interesting questions about garbage collection and persistent executable blocks. Some objects, such as instances of SortedCollection, may contain a reference to an executable block and this persistent reference may be difficult to update and may hold references to other objects that otherwise could be removed by the garbage collection process.

Simple vs. Complex Blocks

The first issue raised by the question is the distinction between a “simple block” and a “complex block.” In Smalltalk a block may reference variables outside the block and, if it does so, then the virtual machine needs to create a more complex data structure to capture the referenced objects (making it a “complex” block). If the block does not reference any variables outside its immediate scope (beyond the block arguments and block temporaries), then the virtual machine can typically use a much simpler implementation (making it a “simple” block).

Because of the extra overhead, complex blocks tend to take more room and be slower to execute. Thus, a common performance tuning activity is replacing complex blocks with simple blocks when possible, and this can occur in application code as well as vendor-supplied libraries. For example, in GemStone/S 64 Bit version 2.x, the implementation of SortedCollection>>#’addAll:’ is essentially the following:

aCollection do: [:each | self add: each].

The problem with this implementation is that the block references self, making it a complex block. In version 3.x the implementation changed (influenced in part by Paul Baumann’s work), and the code is essentially the following:

aCollection accompaniedBy: self do: [:me :each | me add: each].

The block in this code is a simple block since it does not reference self (or anything outside the block), and performance improved.

While this is all very interesting (and in some cases quite useful), I think that the underlying garbage collection problem is not really with simple vs. complex blocks, but with something else.

Class References in Blocks

A code block in a method will contain a reference to the method and thus to a class (or metaclass for a class-side method). Once the code block is persistent, there is a hard reference to the class (and its class variables, class instance variables, and pool variables). Changes to the class schema will create a new “version” of the class but the old version will remain and be referenced by the code block. Even if the old class version is removed from the ClassHistory and all instances of the old version are migrated to the new version, until the code block is replaced in the persistent object, there will be a hard reference to the old class version and its space cannot be reclaimed by the garbage collection process.

To see how this works consider the method SortedCollection>>#’_defaultBlock’:

| block |
block := SortedCollection new _defaultBlock. "anExecBlock2"
block _debugSourceForBlock. "'[ :a :b | a <= b ]'"
block method. "aGsNMethod"
block method isMethodForBlock. "true"
block method inClass. "SortedCollection"

Note that this situation exists whether or not the block is simple or complex and whether or not the block comes from an instance-side method or a class-side method. Thus, this is an orthogonal discussion or a unique concern.

Alternatively, note how an equivalent block can be created by evaluating a String, but does not contain a reference to a Class:

| block |
block := '[ :a :b | a <= b ]' evaluate. "anExecBlock"
block _debugSourceForBlock. "'[ :a :b | a <= b ]'"
block method. "aGsNMethod"
block method isMethodForBlock. "true"
block method inClass. "nil"

So, as suggested one could strip the class reference with the following code:

^ aBlock _sourceString evaluate

Alternatively, one could avoid the private method (#’_sourceString’) by making the String explicit:

^ '[ :a :b | a <= b ]' evaluate

While this avoids the private method, it hides the code in a string making it much more difficult for tools to recognize that this is code (and catch compile errors, support senders/implementors, provide for refactorings, etc.). Further, each of these approaches will create a new instance each time it is called, creating unnecessary objects.

As Dale noted in the email chain referenced above, we could modify the compiler so that a block did not know the class in which it was created. This might improved the garbage collection situation, but would make it more difficult to debug code and track a block back to its source. And, of course, you would need to wait till that feature was added to the product.

One way to address these issues is to create a new class with no instance variables (so that the schema would never change) and use class-side methods to return the desired blocks. The method that would otherwise hold the code block would use a level of indirection to return the actual block. If a code block needed to change, you would create a new class-side method in the special class and edit the indirecting method (so that references to the old code block could be found and modified). The original method would remain as long as there were references to it.

At the cost of a level of indirection (and reduced encapsulation), this gives you actual source code that can be recognized by tools, a single instance, and no stray references from the class (since there is only one class version).

[Note: I found this in “Drafts” and decided to publish it even though the original question was a couple years ago.]


A video of my presentation on GemStone/S and SQL is available here. Unfortunately, the audio is not very strong.

GemStone/S has a configuration STN_TRAN_FULL_LOGGING that, if set to TRUE, causes all transactions to be recorded to the redo-log. In this mode a backup plus transaction logs can be used to recover if the extents are lost. If the configuration is set to FALSE, small transactions are still logged but large transactions cause a “checkpoint” in which all data is written to the extents before the transaction completes. In this mode recovery from a crash is possible if all the extents and the most recent transaction log are available (older logs are typically deleted automatically in this mode). The logs themselves are not sufficient to allow recovery from a backup in partial logging mode, so partial logging mode is inappropriate for production (but handy sometimes for development since old logs are typically deleted automatically).

In some applications or situations it would be nice to segregate transactions into those that should be logged for crash recovery and those that can be easily recreated if needed (and avoid the overhead of time and space in writing to the log). For example, it is common to do a bulk-load of new data before it is needed by the application. If the system crashes, we’d prefer to recover the live data quickly and reload the new data later. With Oracle one can specify a table as being NOLOGGING and then do a series of “Direct-Load INSERT” commands that bypass the rollback/replay log. A key limitation with this capability, however, is that it does not support referential integrity. Likewise, with SQL Server one can do a BULK INSERT that will, under some circumstances, have minimal logging. By default, this will leave the constraint on the table marked as not-trusted.

For a relational database, a foreign key is a value like any other (typically an integer or string) and if a JOIN fails to find a match then the matching data is NULL or the row is ignored. In GemStone/S, an object reference is guaranteed to be valid and there is no easy way to handle data that lacks referential integrity. Thus, if an object is visible in the database (e.g., from a bulk load), then any session can, in a transaction, create a reference to it. If the reference is in the transaction (redo) log, then the object must itself be in the transaction log. Therefore, because in GemStone/S referential integrity is required, not optional, the relational approach to avoiding the transaction log is not safe.

The relational solutions described above for reducing transaction log activity allow for a bulk load of data that can, eventually (following a full back, recreation of indexes, and reapplication of any constraints) be equivalent to all other data. This fits an application upgrade (bulk load) use case quite well.

Another possible use case is short-lived data that is to be shared by multiple sessions but will not be saved over the long-term. One example is transient data where a summary is to be kept but the raw data discarded after the analysis. Another example is information about logged-in sessions with in-progress activity (such as Seaside session state) that could be safely lost in a system crash (presumably the users would need to start over anyway). This use-case does not present the same referential integrity concerns and GemStone/S is developing a feature that is intended to address this use-case: a SymbolDictionary in Globals named #’NotTranloggedGlobals’. The idea is that any object referenced from this root would not be recorded in the transaction log and should not be referenced from any other path. At present this is still a work-in-progress and customers are advised to not use it.

Periodically the International Earth Rotation Service determines that Coordinated Universal Time needs to be adjusted to match the mean solar day. This is because the earth doesn’t rotate at a constant speed and in general takes slightly longer than 86,400 seconds. These adjustments have been done through adding (or, in theory, removing) a second every few years. The next leap second insertion is scheduled for June 30th, 2015 at 23:59:60 UTC. How does GemStone/S handle that?

GemStone gets time from the host OS (Unix Time) and is typically described as the number of seconds that have elapsed since 00:00:00 Coordinated Universal Time (UTC), Thursday, 1 January 1970. This definition is correct so long as we assume that each and every day has exactly 86,400 seconds. If you take into account the 27 leap seconds that will have been added as of June 30, 2015, it is more accurate to say that Unix Time is the number of atomic seconds that have elapsed since 00:00:27 UTC, 1 January 1970. The way the adjustment occurs is that when there is a leap second, the Unix time counter takes two atomic seconds to advance by one Unix second.

The impact of this approach is that a record of when something happened will be correct, but the time between two events could be reported as being one or more seconds less than the actual time between the events. For example, Unix time (and GemStone) will report that there are five seconds between June 30 at 23:59:55 and July 1 at: 00:00:00 when in fact in 2015 there were six seconds between those two times.

Whether this matters is an application-level or domain-level problem. If keeping track of the time between events needs to be accurate (e.g., recording some rapid physics event), then relying on Unix time (or GemStone) will not be sufficient. If all that is needed is a timestamp and it is acceptable to have June 30, 2015 23:59:59 UTC last for 2000 milliseconds, then things should be fine.

Great news! The GemStone/S team is becoming an independent company after 3 years as part of VMware. The core engineering team, including me, along with Norm Green and Dan Ware, have formed GemTalk Systems.

For more information, visit our company web site at

At GemTalk Systems, we’re the people who built the product. We’re excited to become a company with a dedicated focus on Smalltalk, GemStone/S and allied initiatives. You’ll see an increase in innovation on the product, and customers will see a seamless transition in support.

GemTalk will be at the STIC conference (Phoenix in June). Plan to visit us there!

Our email addresses are changing, too. They are all in the form of But the old email addresses will continue to work for a few months.

If you have questions about this exciting transition, feel free to contact me.

“SpringSource has pledged to continue to support GemStone’s current customer base, as well as the company’s other products, such as its software for running distributed Smalltalk applications, GemStone/S.” — PCWorld

See the press release for details.

In this video from chapter 14 of Learning Web Development with Seaside we look at the Monticello source code management system. Monticello allows us to save source code outside of our Smalltalk environment.

We start by launching the Seaside One-Click Experience and opening a Monticello Browser. We click the ‘Add Package’ button and provide a package name. Next we add a repository to the package. We will use a local directory for the repository (I had created a directory named ‘Monticello’ in my home directory). Now we click the ‘Save’ button to save the LosBoquitas package to the repository. We provide a comment and see that we now have a package. When we look at the file system we see that there is a file in the Monticello directory that contains a version of our package.

We would like to understand the package contents. Make a copy of the package and rename the copy so that it has a ‘.zip’ extension. Use the operating system tools to unzip the package and we can see that there are various files and a directory. We can view the package file in a text editor and we can view the vresion file in a text editor. We see that the version file is a list of name/value pairs describing the current version. In the snapshot directory there is a ‘’ file that contains the code for our package.

Now we can make a change to one of the methods in our package and see how Monticello marks changed files. We can view the changes in a Patch Browser. The green shows our current code and the red shows the prior code.

We now save our Smalltalk environment and quit.

For more information go to

In this video from chapter 12 of Learning Web Development with Seaside we look at how Seaside can support user login. We start by launching the Seaside One-Click Experience and creating a new domain model class to represent a user. We use the class refactoring tool to add instance variable accessors. Note that the accessor for the ‘name’ instance variable is given the method name ‘name1’. This is because it overrides a method in Object named ‘name’. Since we want the override, we will add our method and remove the provided one.

Because it is a poor practice to save passwords, we will store a hash of the password. We provide a method to verify passwords by comparing the hash of a provided string. We add an initialize method to set reasonable default values and we add a comparing method to support sorting.

On the class side we define a class instance variable to hold a list of users and we define a class-side method to return the list. This method includes a lazy initializer to set up a default initial user. Finally, we add a class-side method to lookup a user based on an id and password.

Seaside provides a way to keep information about a web session. We subclass WASession to provide a class with a ‘user’ instance variable. We use the refactoring tool to create accessors. When we register our main component as a Seaside application, we can specify the class to be used for the session.

Now we will create a login component to collect the user ID and password. We define a render method that simply shows the class name so we can see that it is being called correctly. Now we need to use the new component. We will change the main page’s sidebar so that it supports the login/logout feature. To do this we will refactor the render sidebar method so that the code dealing with the home anchor is in a separate method.

Instead of using cut-and-paste, Smalltalk gives us an ‘extract method’ refactoring in which we can select lines of a method and define a new method to contain those lines. We give the new method a name #’renderHomeAnchorOn:’ and the refactoring tool shows us the proposed changes. Whe we accept the changes we see the changes. We can then extract the ‘Events’ anchor as well. I’ll change the formatting a bit.

Now we can create a new method to rendre a login link. We then refactor the render sidebar method to call our new login anchor. We can view the appliaction in a web browser and see that it shows the login link and when you click on the link it shows the login component. Now we will make the login component useful.

We will create a method to render the user ID and allow text input to edit the userID. When we try to save the method we discover that the instance variable is not properly named. After that is fixed, we can paste and save the edit method. We rename and modify this method to create a div to display and edit the password. We create a warning method to return a string describing the current state, logged in, need to log in, and login failed. When we try to save the method Smalltalk alerts us to a misspelled message send.

We add a render method to show the warning. We add a render method to render a submit button named ‘Login’ that calls the login method. We add a method to render a form with the user edit div, the password edit div, the warning div, and the login div. Finally, we edit the render content method to call the render form method.

When we view the login component in a web browser, we see that the various elements exist, but the layout is not very good. We modify our CSS so that all forms are displayed as a table. This fixes up the formatting of the login component. when we refresh the page we find that focus is set to the password element. We want it on the user ID element and see in the code that we copied the ‘setFocus’ message from the user method. After removing that message we return to the web browser and find that focus is properly set to the user field.

clicking the login button gives an error because the login method is not yet defined. We create a method to lookup the user based on the user and password entered. We call our earlier method to lookup among the defined users. When we try the application again we give a wrong user/password and see that it tells us that the login failed. Note also that the password is displayed in clear text. We fix this by going back to the render password method and changing the HTML element. When we refresh the browser we see that the password is obscured.

We can try the correct user ID (‘admin’) without a password and see that the login failed. When we try the correct user ID and password we are told that we are logged in as Site Administrator.

Now we will modify the render content method so that if no user exists, we will invite the login. If a user is logged in, then we show the user. This lets the user know that they are logged in.

Now we go back to our main application and add an anchor that has a callback that clears the session user and label it ‘Logout’. We modify the login anchor to remove the break element. Now we create a render user method that checks whether a user exists. If no user exists, then we offer the login option. If a user does exist, then we offer the logout option. Finally we modify the sidebar render method so that instead of rendering a login we render the user. We try the application in a web browser and see that when we are properly logged in we get a logout link.

Now we are going to limit some functionality to logged in users. We go to the schedule list component and make the add link limited to logged in users. We go back to the web browser and verify that the functionality works as expected. Next we modify the table initialization method so that the ‘action’ column (that supports the delete function) is available only to logged in users. We try the application and see that when we are not logged in we don’t see the delete anchor. When we are logged in we do see the delete anchor.

We save our code and quit the Seaside One-Click Experience.

For more information go to