For more information, visit our company web site at http://gemtalksystems.com.

At GemTalk Systems, we’re the people who built the product. We’re excited to become a company with a dedicated focus on Smalltalk, GemStone/S and allied initiatives. You’ll see an increase in innovation on the product, and customers will see a seamless transition in support.

GemTalk will be at the STIC conference (Phoenix in June). Plan to visit us there!

Our email addresses are changing, too. They are all in the form of first.last@gemtalksystems.com. But the old email addresses will continue to work for a few months.

If you have questions about this exciting transition, feel free to contact me.

#!/bin/bash
# cleanup log files
cd /opt/gemstone/log
(ls -t *pcmon.log | head -n 2; ls *pcmon.log) | \
    sort | uniq -u | xargs rm
(ls -t *pagemanager.log | head -n 2; ls *pagemanager.log) | \
    sort | uniq -u | xargs rm
(ls -t *symbolgem.log | head -n 2; ls *symbolgem.log) | \
    sort | uniq -u | xargs rm
# do backup
topaz -l -T 50000 << EOF
output push backup.out
errorCount
send SystemRepository markForCollection
send SystemRepository reclaimAll
send SystemRepository startNewLog
run
SystemRepository fullBackupCompressedTo: 
 '/opt/gemstone/backups/backup-' , 
 (DateTime now asSeconds // 60) printString.
%
logout
errorCount
exit
EOF
# cleanup backups
cd /opt/gemstone/backups
(ls -t backup* | head -n 2; ls backup*) | sort | uniq -u | xargs rm
if [ "2" -le "`find . -mtime -2 -name 'backup*' | wc -l`" ]; then 
 cd /opt/gemstone/data
 find . -mtime +2 -name 'tranlog*' | xargs rm
fi

GLASS uses FastCGI to route requests to long-running server processes (typically a Topaz process) that remains logged in to the GemStone database. The Topaz processes can run on a different host from the web server (Apache or whatever), using the ‘ExternalServer’ (discussed here and here). While FastCGI is sometimes thought to be out-of-date, it is more accurate to say that “There is not much development on FastCGI because it is a very stable protocol / application.” Apache now provides a fast cgi module (here), that is commonly described as a replacement to mod_fastcgi, but it doesn’t support the external server configuration used by GLASS.

Getting GLASS to work on Mac OS X 10.8 requires jumping through some hoops, but now that I’ve done it I’ll describe the steps I took (please feel free to suggest alternatives in the comments).

I used the App Store application to find and install Xcode (4.6), then installed the command line tools (from Xcode Preferences->Downloads or from Apple). Next, I installed the latest MacPorts. Then, from a Terminal I entered the following command:

sudo port install mod_fastcgi

This did a full Apache build, but also created the needed modules. Instead of running the MacPorts build of Apache, I copied the new module to the expected directory:

sudo cp /opt/local/apache2/modules/mod_fastcgi.so \
    /usr/libexec/apache2/mod_fastcgi.so

Then I was able to run my usual setup with FastCGI on Mac OS X 10.8!

Fortunately, it is relatively easy to run Topaz on Windows. Open a web browser on http://seaside.gemstone.com/downloads/x86.Windows_NT/ and download GemBuilderC3.1.0.1-x86.Windows_NT.zip (this assumes that you are connecting to a 3.1.0.1 server). Unzip this into a convenient location on your Windows machine (e.g., C:\gemstone\). Open a command shell (Start, All Programs, Accessories, Command Prompt), navigate to the ‘bin’ directory created by unzipping the download, and start ‘topaz’. At this point you can enter the usual Topaz commands and try to login to your server. Following is a copy of the screen when I used Topaz on Windows to login to my database:

C:\GemStone\GemBuilderC3.1.0.1-x86.Windows_NT\bin>topaz
 _____________________________________________________________________________
| GemStone/S64 Object-Oriented Data Management System                         |
| Copyright (C) VMware, Inc. 1986-2012                                        |
| All rights reserved.                                                        |
+-----------------------------------------------------------------------------+
| PROGRAM: topaz, Linear GemStone Interface (Remote Session)                  |
| VERSION: 3.1.0.1, Fri Aug 24 10:24:31 2012                                  |
| BUILD: gss64_3_1_0_x_branch-28937                                           |
| BUILT FOR: Pentium/Windows_NT                                               |
| MODE: 32 bit                                                                |
| RUNNING ON: 1-CPU jfoster-xpvm: Intel CPU, Windows NT 5.1 build 2600 Service|
| Pack 3                                                                      |
| PROCESS ID: 828 DATE: 11/28/2012 11:19:36 Pacific Standard Time             |
|_____________________________________________________________________________|
neither topazini.tpz nor $HOME\topazini.tpz were found
topaz> set user DataCurator pass swordfish
topaz> set gemstone jfoster0
topaz> set gemnet !tcp@10.118.32.15#netldi:10460#task!gemnetobject
topaz> login
[Info]: libssl-3.1.0.1-32.dll: loaded
[11/28/2012 11:20:47.435 Pacific Standard Time]
 gci login: currSession 1 rpc gem processId 30480 OOB keep-alive interval 0
successful login
topaz 1> run
100 factorial printString
%
93326215443944152681699238856266700490715968264381621468592963895217599993229915
608941463976156518286253697920827223758251185210916864000000000000000000000000
topaz 1> logout
topaz> exit

C:\GemStone\GemBuilderC3.1.0.1-x86.Windows_NT\bin>

The things I typed are in bold. The things you need to change are in italics. Specifically, you need to provide the name of your stone (perhaps it is ‘seaside’), the IP address (or hostname if you have an entry in your hosts file) and port number (or service name if you have an entry in your services file) for your NetLDI that will start your gem.

I now have something that works as of 30 August 2012. To see the changes go to my github repository and view the changes to  vcap, vcap-staging, stager, and cloud_controller. Note that the recent refactoring of Cloud Foundry means that instead of isolating changes to one repository, we now make changes to four repositories.

Of course, there may be more efficient ways of doing this, but this is what I’ve found to work for me today!

Update 1: It wasn’t much work to update the earlier work on Cog/Aida. It involved similar changes to vcap, vcap-staging, stager, and cloud_controller.

The currently-approved way to do privileged operations on the Mac is to create a “helper tool” (a small C application) that does the minimal task. A client application (running as a regular user) then can install the tool and (with appropriate authorizations) “bless” it so that launchd can start it as root when it is needed. Apple provides an example of how to install a helper tool (SMJobBless), but the sample helper does not actually do anything in the way of providing a service to the sample application. It seems that there are a number of requests on the web for a more complete sample, and so rather than moving straight to my application I started by enhancing the sample app. You can find the result on github.

Communication is done using Unix domain sockets. That is, rather than listening on a port, the server listens on a file. This allows a safer naming scheme that avoids overlap better than port numbers. It also limits requests to clients on the same machine. Socket programming is reasonable straightforward, but somewhat tedious. Because it is stream-based, one can’t be sure that an entire message has been received so there needs to be code that figures out how many bytes to expect and then waits till they come. Of course, you shouldn’t wait forever, so there are timing issues as well.

If you are building a Mac application that needs a helper tool, then take a look and let us know what is missing…

One of the features of modern operating systems is that each operating system process runs in its own address space and (with help from the hardware) the process is not permitted to read or write outside its assigned area. This protection makes it much less likely that a misbehaving application will harm other applications or even crash your whole machine (as was common on personal computers during the era prior to Windows XP and Mac OS X). This protection comes with the overhead that if two processes want to share data they typically need to communicate through the operating system (typically using TCP/IP, as if they were on separate machines). Processes that want to share a large amount of data are allowed to request a chunk of memory from the operating system that can be shared. Because this is not common, most operating systems are configured to provide only a limited amount of memory for this use and without adjustments to those settings GemStone cannot run.

Each machine where a GemStone process is running will have a Shared Page Cache (SPC) that holds copies of 16 KB pages from the file system extents that make up the repository. The SPC is a chunk of shared memory allocated by a SPC “monitor” and to which other processes attach. When the stone starts it forks a process to be the SPC monitor and then waits for the SPC to become available. If the SPC does not appear within a set time then the stone reports an error and dies. This error will be similar to the following:

The stone was unable to start a cache page server on host '<stone's host>'. 
Reason: The cache monitor connect failed. 
Monitor process (9999) did not start.

The SPC monitor log (a file next to the stone log whose name ends with ‘pcmon.log’) will have the following:

| GemStone could not retrieve the IPC identifier associated with the memory |
| key 9999999. shmget() error = errno=22,EINVAL, Invalid argument (programmer
| error). |
| |
 GemStone could not attach to the shared page cache.
|
 [SpcMon trace]: ... cache creation failed ...
 [SpcMon trace]: ... if the errno is (EINVAL) it is likely because the
 cache size is less than the operating system imposed
 minimum or greater than the operating system maximum.

As suggested in the error message, the cache size is likely greater than the operating system maximum. If this is the first error you get after installing GemStone then it is likely that you have not configured the operating system for adequate shared memory. To see the existing limits, you can ask for a listing of the inter-process communication statistics:

ipcs -l

This provides a list similar to the following (from my Ubuntu 10.04 desktop machine at work):

------ Shared Memory Limits --------
max number of segments = 4096
max seg size (kbytes) = 4592442
max total shared memory (kbytes) = 4592440
min seg size (bytes) = 1

This machine is configured to make 4 GB of RAM available to be used as shared memory, and the maximum segment is also 4 GB. We can also query the current settings with the following command:

sysctl -a 2> /dev/null | grep kernel.shm

On my machine this reports the following:

kernel.shmmax = 4702660608
kernel.shmall = 1148110
kernel.shmmni = 4096

These numbers mean the same thing as shown above, but the first two lines use different values. While ipcs reports both the max segment size and the max overall shared memory as kbytes, sysctl reports the max segment size (kernel.shmmax) as bytes and the max overall shared memory (kernel.shmall) as 4 KB pages.

These settings are configured at boot time by reading /etc/sysctl.conf and applying defaults if no values are provided. To see the configured values, enter the following:

grep kernel.shm /etc/sysctl.conf

In my case I see the following lines:

kernel.shmmax = 4702660608
kernel.shmall = 1148110

If you edit /etc/sysctl.conf (as root using sudo) and reboot your machine then the new values should take effect. If you want to make changes without rebooting your machine, use something like the following:

sudo sysctl -w kernel.shmmax=4702660608

Once you have things properly configured then the SPC monitor should be able to allocation a SPC and the stone should start. To see the memory use, you can enter the following

ipcs

On my machine the output includes the following:

------ Shared Memory Segments --------
key shmid owner perms bytes nattch status 
...
0x00000000 1409046 jfoster 600 393216 2 dest 
0xe50206db 8683543 jfoster 660 529203200 8 locked
...

There are a number of segments of 384 KB that are marked to be destroyed (dest), and one segment of about 500 MB. My configuration file includes the following:

SHR_PAGE_CACHE_SIZE_KB = 500000;

The pcmon.log file shows that the number of pages requested:

Number of pages 31250.

Since each page is 16 KB, this would request 512000000 bytes for pages. It turns out that there are other data structures that need to be shared, including the statistics captured by statmonitor, and the memory requested will be slightly larger than the configured amount. This means that you should set shmmax and shmall to be larger than your SHR_PAGE_CACHE_SIZE_KB setting (or ask for less than the configured maximum).

The GemStone/S 64 Bit Install Guide has a good discussion of how to calculate shared memory needs, but at present it contains an error in that the shmmax setting is incorrectly shown as being the same as the shmall setting. Remember that shmmax is in bytes and shmall is in pages (typically 4 KB).