You are currently browsing the category archive for the ‘Uncategorized’ category.

A question on the GLASS mailing list raised some interesting questions about garbage collection and persistent executable blocks. Some objects, such as instances of SortedCollection, may contain a reference to an executable block and this persistent reference may be difficult to update and may hold references to other objects that otherwise could be removed by the garbage collection process.

Simple vs. Complex Blocks

The first issue raised by the question is the distinction between a “simple block” and a “complex block.” In Smalltalk a block may reference variables outside the block and, if it does so, then the virtual machine needs to create a more complex data structure to capture the referenced objects (making it a “complex” block). If the block does not reference any variables outside its immediate scope (beyond the block arguments and block temporaries), then the virtual machine can typically use a much simpler implementation (making it a “simple” block).

Because of the extra overhead, complex blocks tend to take more room and be slower to execute. Thus, a common performance tuning activity is replacing complex blocks with simple blocks when possible, and this can occur in application code as well as vendor-supplied libraries. For example, in GemStone/S 64 Bit version 2.x, the implementation of SortedCollection>>#’addAll:’ is essentially the following:

aCollection do: [:each | self add: each].

The problem with this implementation is that the block references self, making it a complex block. In version 3.x the implementation changed (influenced in part by Paul Baumann’s work), and the code is essentially the following:

aCollection accompaniedBy: self do: [:me :each | me add: each].

The block in this code is a simple block since it does not reference self (or anything outside the block), and performance improved.

While this is all very interesting (and in some cases quite useful), I think that the underlying garbage collection problem is not really with simple vs. complex blocks, but with something else.

Class References in Blocks

A code block in a method will contain a reference to the method and thus to a class (or metaclass for a class-side method). Once the code block is persistent, there is a hard reference to the class (and its class variables, class instance variables, and pool variables). Changes to the class schema will create a new “version” of the class but the old version will remain and be referenced by the code block. Even if the old class version is removed from the ClassHistory and all instances of the old version are migrated to the new version, until the code block is replaced in the persistent object, there will be a hard reference to the old class version and its space cannot be reclaimed by the garbage collection process.

To see how this works consider the method SortedCollection>>#’_defaultBlock’:

| block |
block := SortedCollection new _defaultBlock. "anExecBlock2"
block _debugSourceForBlock. "'[ :a :b | a <= b ]'"
block method. "aGsNMethod"
block method isMethodForBlock. "true"
block method inClass. "SortedCollection"

Note that this situation exists whether or not the block is simple or complex and whether or not the block comes from an instance-side method or a class-side method. Thus, this is an orthogonal discussion or a unique concern.

Alternatively, note how an equivalent block can be created by evaluating a String, but does not contain a reference to a Class:

| block |
block := '[ :a :b | a <= b ]' evaluate. "anExecBlock"
block _debugSourceForBlock. "'[ :a :b | a <= b ]'"
block method. "aGsNMethod"
block method isMethodForBlock. "true"
block method inClass. "nil"

So, as suggested one could strip the class reference with the following code:

^ aBlock _sourceString evaluate

Alternatively, one could avoid the private method (#’_sourceString’) by making the String explicit:

^ '[ :a :b | a <= b ]' evaluate

While this avoids the private method, it hides the code in a string making it much more difficult for tools to recognize that this is code (and catch compile errors, support senders/implementors, provide for refactorings, etc.). Further, each of these approaches will create a new instance each time it is called, creating unnecessary objects.

As Dale noted in the email chain referenced above, we could modify the compiler so that a block did not know the class in which it was created. This might improved the garbage collection situation, but would make it more difficult to debug code and track a block back to its source. And, of course, you would need to wait till that feature was added to the product.

One way to address these issues is to create a new class with no instance variables (so that the schema would never change) and use class-side methods to return the desired blocks. The method that would otherwise hold the code block would use a level of indirection to return the actual block. If a code block needed to change, you would create a new class-side method in the special class and edit the indirecting method (so that references to the old code block could be found and modified). The original method would remain as long as there were references to it.

At the cost of a level of indirection (and reduced encapsulation), this gives you actual source code that can be recognized by tools, a single instance, and no stray references from the class (since there is only one class version).

[Note: I found this in “Drafts” and decided to publish it even though the original question was a couple years ago.]

 

Advertisements

A video of my presentation on GemStone/S and SQL is available here. Unfortunately, the audio is not very strong.

GemStone/S has a configuration STN_TRAN_FULL_LOGGING that, if set to TRUE, causes all transactions to be recorded to the redo-log. In this mode a backup plus transaction logs can be used to recover if the extents are lost. If the configuration is set to FALSE, small transactions are still logged but large transactions cause a “checkpoint” in which all data is written to the extents before the transaction completes. In this mode recovery from a crash is possible if all the extents and the most recent transaction log are available (older logs are typically deleted automatically in this mode). The logs themselves are not sufficient to allow recovery from a backup in partial logging mode, so partial logging mode is inappropriate for production (but handy sometimes for development since old logs are typically deleted automatically).

In some applications or situations it would be nice to segregate transactions into those that should be logged for crash recovery and those that can be easily recreated if needed (and avoid the overhead of time and space in writing to the log). For example, it is common to do a bulk-load of new data before it is needed by the application. If the system crashes, we’d prefer to recover the live data quickly and reload the new data later. With Oracle one can specify a table as being NOLOGGING and then do a series of “Direct-Load INSERT” commands that bypass the rollback/replay log. A key limitation with this capability, however, is that it does not support referential integrity. Likewise, with SQL Server one can do a BULK INSERT that will, under some circumstances, have minimal logging. By default, this will leave the constraint on the table marked as not-trusted.

For a relational database, a foreign key is a value like any other (typically an integer or string) and if a JOIN fails to find a match then the matching data is NULL or the row is ignored. In GemStone/S, an object reference is guaranteed to be valid and there is no easy way to handle data that lacks referential integrity. Thus, if an object is visible in the database (e.g., from a bulk load), then any session can, in a transaction, create a reference to it. If the reference is in the transaction (redo) log, then the object must itself be in the transaction log. Therefore, because in GemStone/S referential integrity is required, not optional, the relational approach to avoiding the transaction log is not safe.

The relational solutions described above for reducing transaction log activity allow for a bulk load of data that can, eventually (following a full back, recreation of indexes, and reapplication of any constraints) be equivalent to all other data. This fits an application upgrade (bulk load) use case quite well.

Another possible use case is short-lived data that is to be shared by multiple sessions but will not be saved over the long-term. One example is transient data where a summary is to be kept but the raw data discarded after the analysis. Another example is information about logged-in sessions with in-progress activity (such as Seaside session state) that could be safely lost in a system crash (presumably the users would need to start over anyway). This use-case does not present the same referential integrity concerns and GemStone/S is developing a feature that is intended to address this use-case: a SymbolDictionary in Globals named #’NotTranloggedGlobals’. The idea is that any object referenced from this root would not be recorded in the transaction log and should not be referenced from any other path. At present this is still a work-in-progress and customers are advised to not use it.

Periodically the International Earth Rotation Service determines that Coordinated Universal Time needs to be adjusted to match the mean solar day. This is because the earth doesn’t rotate at a constant speed and in general takes slightly longer than 86,400 seconds. These adjustments have been done through adding (or, in theory, removing) a second every few years. The next leap second insertion is scheduled for June 30th, 2015 at 23:59:60 UTC. How does GemStone/S handle that?

GemStone gets time from the host OS (Unix Time) and is typically described as the number of seconds that have elapsed since 00:00:00 Coordinated Universal Time (UTC), Thursday, 1 January 1970. This definition is correct so long as we assume that each and every day has exactly 86,400 seconds. If you take into account the 27 leap seconds that will have been added as of June 30, 2015, it is more accurate to say that Unix Time is the number of atomic seconds that have elapsed since 00:00:27 UTC, 1 January 1970. The way the adjustment occurs is that when there is a leap second, the Unix time counter takes two atomic seconds to advance by one Unix second.

The impact of this approach is that a record of when something happened will be correct, but the time between two events could be reported as being one or more seconds less than the actual time between the events. For example, Unix time (and GemStone) will report that there are five seconds between June 30 at 23:59:55 and July 1 at: 00:00:00 when in fact in 2015 there were six seconds between those two times.

Whether this matters is an application-level or domain-level problem. If keeping track of the time between events needs to be accurate (e.g., recording some rapid physics event), then relying on Unix time (or GemStone) will not be sufficient. If all that is needed is a timestamp and it is acceptable to have June 30, 2015 23:59:59 UTC last for 2000 milliseconds, then things should be fine.

Great news! The GemStone/S team is becoming an independent company after 3 years as part of VMware. The core engineering team, including me, along with Norm Green and Dan Ware, have formed GemTalk Systems.

For more information, visit our company web site at http://gemtalksystems.com.

At GemTalk Systems, we’re the people who built the product. We’re excited to become a company with a dedicated focus on Smalltalk, GemStone/S and allied initiatives. You’ll see an increase in innovation on the product, and customers will see a seamless transition in support.

GemTalk will be at the STIC conference (Phoenix in June). Plan to visit us there!

Our email addresses are changing, too. They are all in the form of first.last@gemtalksystems.com. But the old email addresses will continue to work for a few months.

If you have questions about this exciting transition, feel free to contact me.

“SpringSource has pledged to continue to support GemStone’s current customer base, as well as the company’s other products, such as its software for running distributed Smalltalk applications, GemStone/S.” — PCWorld

See the press release for details.

In this video from chapter 14 of Learning Web Development with Seaside we look at the Monticello source code management system. Monticello allows us to save source code outside of our Smalltalk environment.

We start by launching the Seaside One-Click Experience and opening a Monticello Browser. We click the ‘Add Package’ button and provide a package name. Next we add a repository to the package. We will use a local directory for the repository (I had created a directory named ‘Monticello’ in my home directory). Now we click the ‘Save’ button to save the LosBoquitas package to the repository. We provide a comment and see that we now have a package. When we look at the file system we see that there is a file in the Monticello directory that contains a version of our package.

We would like to understand the package contents. Make a copy of the package and rename the copy so that it has a ‘.zip’ extension. Use the operating system tools to unzip the package and we can see that there are various files and a directory. We can view the package file in a text editor and we can view the vresion file in a text editor. We see that the version file is a list of name/value pairs describing the current version. In the snapshot directory there is a ‘source.st’ file that contains the code for our package.

Now we can make a change to one of the methods in our package and see how Monticello marks changed files. We can view the changes in a Patch Browser. The green shows our current code and the red shows the prior code.

We now save our Smalltalk environment and quit.

For more information go to http://seaside.gemstone.com/tutorial.html.

In this video from chapter 12 of Learning Web Development with Seaside we look at how Seaside can support user login. We start by launching the Seaside One-Click Experience and creating a new domain model class to represent a user. We use the class refactoring tool to add instance variable accessors. Note that the accessor for the ‘name’ instance variable is given the method name ‘name1’. This is because it overrides a method in Object named ‘name’. Since we want the override, we will add our method and remove the provided one.

Because it is a poor practice to save passwords, we will store a hash of the password. We provide a method to verify passwords by comparing the hash of a provided string. We add an initialize method to set reasonable default values and we add a comparing method to support sorting.

On the class side we define a class instance variable to hold a list of users and we define a class-side method to return the list. This method includes a lazy initializer to set up a default initial user. Finally, we add a class-side method to lookup a user based on an id and password.

Seaside provides a way to keep information about a web session. We subclass WASession to provide a class with a ‘user’ instance variable. We use the refactoring tool to create accessors. When we register our main component as a Seaside application, we can specify the class to be used for the session.

Now we will create a login component to collect the user ID and password. We define a render method that simply shows the class name so we can see that it is being called correctly. Now we need to use the new component. We will change the main page’s sidebar so that it supports the login/logout feature. To do this we will refactor the render sidebar method so that the code dealing with the home anchor is in a separate method.

Instead of using cut-and-paste, Smalltalk gives us an ‘extract method’ refactoring in which we can select lines of a method and define a new method to contain those lines. We give the new method a name #’renderHomeAnchorOn:’ and the refactoring tool shows us the proposed changes. Whe we accept the changes we see the changes. We can then extract the ‘Events’ anchor as well. I’ll change the formatting a bit.

Now we can create a new method to rendre a login link. We then refactor the render sidebar method to call our new login anchor. We can view the appliaction in a web browser and see that it shows the login link and when you click on the link it shows the login component. Now we will make the login component useful.

We will create a method to render the user ID and allow text input to edit the userID. When we try to save the method we discover that the instance variable is not properly named. After that is fixed, we can paste and save the edit method. We rename and modify this method to create a div to display and edit the password. We create a warning method to return a string describing the current state, logged in, need to log in, and login failed. When we try to save the method Smalltalk alerts us to a misspelled message send.

We add a render method to show the warning. We add a render method to render a submit button named ‘Login’ that calls the login method. We add a method to render a form with the user edit div, the password edit div, the warning div, and the login div. Finally, we edit the render content method to call the render form method.

When we view the login component in a web browser, we see that the various elements exist, but the layout is not very good. We modify our CSS so that all forms are displayed as a table. This fixes up the formatting of the login component. when we refresh the page we find that focus is set to the password element. We want it on the user ID element and see in the code that we copied the ‘setFocus’ message from the user method. After removing that message we return to the web browser and find that focus is properly set to the user field.

clicking the login button gives an error because the login method is not yet defined. We create a method to lookup the user based on the user and password entered. We call our earlier method to lookup among the defined users. When we try the application again we give a wrong user/password and see that it tells us that the login failed. Note also that the password is displayed in clear text. We fix this by going back to the render password method and changing the HTML element. When we refresh the browser we see that the password is obscured.

We can try the correct user ID (‘admin’) without a password and see that the login failed. When we try the correct user ID and password we are told that we are logged in as Site Administrator.

Now we will modify the render content method so that if no user exists, we will invite the login. If a user is logged in, then we show the user. This lets the user know that they are logged in.

Now we go back to our main application and add an anchor that has a callback that clears the session user and label it ‘Logout’. We modify the login anchor to remove the break element. Now we create a render user method that checks whether a user exists. If no user exists, then we offer the login option. If a user does exist, then we offer the logout option. Finally we modify the sidebar render method so that instead of rendering a login we render the user. We try the application in a web browser and see that when we are properly logged in we get a logout link.

Now we are going to limit some functionality to logged in users. We go to the schedule list component and make the add link limited to logged in users. We go back to the web browser and verify that the functionality works as expected. Next we modify the table initialization method so that the ‘action’ column (that supports the delete function) is available only to logged in users. We try the application and see that when we are not logged in we don’t see the delete anchor. When we are logged in we do see the delete anchor.

We save our code and quit the Seaside One-Click Experience.

For more information go to seaside.gemstone.com/tutorial.

In this video of Chapter 11 of Learning Web Development with Seaside we create a form to collect information from the user. We start by launching the Seaside One-click Experience and then defining a new web component to edit events. We create a render content method that initially just shows the class name. We add an initialize method that saves a passed-in event. Now, on the class side we create a constructor method that takes an event and returns a newly initialize component.

Now we go to an earlier web component that lists all the events and add a method to call our new editor component. We modify one of the report columns so that instead of simply displaying information about the event it offers an opportunity to edit the event. When we view the event list in a web browser, we see that data in the ‘What’ column now displays as a link. Clicking on that link takes us to the editor, which simply displays its class name.

Now we will add true editing. We modify the render content method to show a form with labels and entry fields. (If you don’t like the table for styling, hang on a minute…) Using the web browser, we can edit fields in an event and see that they are modified.

Now, we will refactor our form to avoid the use of a table for layout. We create small methods for each element to be edited. Note that instead of table rows we have a div enclosing our elements. Rather than typing each method we are copying them from the tutorial. Finally we modify the render method to call each of our elements.

When we view the form in a browser, we see that the layout is no longer based on a table. To fix the layout, add some CSS styling to show the event editor as a table and note that the layout is now better. We find that the two buttons are treated as one column. We can add an empty field that causes the buttons to move over a column. If we want the buttons in separate columns we need to put them in a span. When that is done they display more evenly.

Now we modify our application to allow the creation of a new event. We add an ‘Add Event’ anchor to the event list, and see that it shows in the web browser. Clicking on the link gives an error since our callback message is not understood. We create the add method so that it creates a new event, allows the user to edit it, and then if the edit was saved the new event is added to the event list. We can try it out and see that saving a new event does give us a new event. Observe that we are using the new component for both editing existing events and collecting information on new events, yet the component does not know how it is being used. This is an example of good encapsulation and code reuse.

Note how we are using the result of the call, which answers true if the ‘Save’ button was clicked, to decide whether or not to add the new event to the saved event list. We can simplify the edit method since we don’t really need to alert the user to which button was pressed.

As we experiment with adding events, we discover that changing the date does not seem to work. Let’s go investigate that problem. Looking closely at the when field we see that we render the date/time but do not have a callback for any edits to the field. Internally this field is a DateAndTime, but it is being displayed as a string. Let’s fix that problem now. We will add an instance variable to our editor to hold another component, a date/time selector. When the component is initialized, we add a sub-component to collect the date/time. Then when we offer the field to be edited, we use the subcomponent rather than a text entry field. Remember to add a #’children’ method to let the Seaside framework know that we have added another component to the page. Finally, we modify the save callback so that the value is copied from the subcomponent to the model.

Now we would like to demonstrate some other form elements. Rather than having ‘who’ be a text entry field, we will modify it to use a drop-down list. We make a similar change for the ‘what’ portion of the event. We will show the where as a text area and modify the CSS to layout the text area. We can modify the report column to show a string for the date/time.

In order to demonstrate checkboxes, radio buttons, and some JavaScript interaction with CSS, we will add an attribute to LBEvent to keep track of whether a game is home or away. We add a ‘gameType’ instance variable and accessors. We then add some instance variables to the event editor to capture some characteristics of the event. We add CSS for hiding a form element.

Now we are going to add a method to render the ‘isGame’ and ‘gameType’ fields. Smalltalk alerts us to the fact that these methods do not yet exist and we confirm the selectors. We add the new methods, one with some JavaScript for an on-click event.

When we run the application from a web browser we discover a bug in the render when method. We should be using the message #’with:’ rather than #’value:’ to the span element. Fixing that in a debugger lets the form display. We can now view the editor and see various form elements. The who field is a drop-down list. The what field is a scrolling list. The when field is a subcomponent with a series of fields. The where field is a multi-line text area. Changing fields, including the date, makes appropriate changes to the event.

When we refresh the application we see that the ‘is game’ checkbox shows and hides the game type field. We modify the save method to store a value for the game type field, either home, away, or nil.

We save our new code and quit the Seaside One-Click Experience.

For more information go to seaside.gemstone.com/tutorial.

Categories

Advertisements