MCP PWB/Copy Utility

PWB/COPY is a utility for Unisys MCP systems that recursively copies whole directories of source files from MCP file systems to SMB/CIFS shared directories. It converts MCP source to standard text files in PWB (Programmers Workbench) format.

This project is part of the UNITE Source Code Repository hosted on SourceForge.

Description

PWB/Copy converts directories of source files residing in Unisys MCP file systems to PWB format and transfers the converted files to SMB/CIFS shared directories on other networked systems.

PWB format is an ordinary Microsoft Windows text file (i.e., with carriage-return/line-feed delimiters), but with the text lines formatted in the specific way required by the Windows-based Programmers Workbench editor (formerly known as NX/EDIT). Essentially, text lines in these files are formatted identically to source records in the MCP environment, including trailing spaces and the positioning of any sequence number or patchmark fields. The PWB editor is very fussy about this format, and will refuse to edit a file that does not meet its requirements. It does have an import tool, however, that will convert most text file formats into the form it requires.

SMB/CIFS is commonly known as "Windows networking," and is the protocol used by Windows to support shared directories and mapped drives. It is also the protocol used by Samba on Linux/Unix (*nix) systems.

PWB/Copy provides much the same capability as the Unisys MCP File Copier utility that is part of the standard software release. One big difference is that PWB/Copy runs from within the MCP environment, while the File Copier utility, and its command line cousin, MCPCopy.exe, are Windows programs. Another big difference (which is what led to PWB/Copy's development) is the way in which file names that refer to both a file and a directory are handled:

• In the MCP file system, unlike most others, a given file name can refer to both a file and a directory.¹
• In order to transfer files using MCP File Copier or MCPCopy.exe, an MCP directory containing the files must be defined as a shared directory in Client Access Services. Because these are Windows programs, they use the Windows APIs, which are limited to dealing with a given name as either a file or a directory, but not both.
• To accommodate the limitation that most other systems have in dealing with a name as either a file or a directory, MCP Client Access Services handles names that are both a file and a directory by exposing the file to the network, but hiding the existence of the directory. Thus, if there are three files: A/B, A/B/1, and A/B/2, and A/= is a shared directory, remote clients connecting to this share will be able to see file A/B, but not the other two, since they are under the directory A/B/=.
• The sad part about this situation is that when an MCP source file is converted to a text file for use on a non-MCP system, the resulting file is usually named with an extension indicating its type. PWB uses a naming convention that includes a suffix of "_m" (for MCP) on all extensions, e.g., ".alg_m" for Algol, ".c85_m" for COBOL-85, and ".dat_m" for data files.
• If the three files where all COBOL-85 source files and could be accessed by MCP File Copier, they would end up on the remote system named A/B.c85_m, A/B/1.c85_m, and A/B/2.c85_m. Note that there is no file/directory naming conflict anymore—what was the file A/B on the MCP is now A/B.c85_m, leaving A/B to be a valid directory name in the remote file system. Thus, the problem with names that refer to both a file and a directory is not in how the files will be ultimately named on the remote system, it is in the remote system's inability to deal with the dual use of the name through SMB/CIFS.
• Since PWB/Copy runs in the MCP environment, it can deal with names that refer to both a file and a directory. It appends a PWB-style extension to the names of files that it converts, and thus the names of the converted files as transferred to the remote system will not conflict with any of the directory names coming from the MCP environment.
• Additionally, since PWB/Copy runs in the MCP, it can transfer converted files to any type of system that supports SMB/CIFS shared directories. MCP File Copier and MCPCopy.exe are Windows programs—they can't, for example, run in a Linux environment.

See the PWB editor on-line help file for the correspondence between its file name extensions and MCP FILEKIND values, and the discussion of the FILEKIND attribute in the File Attributes Programming Reference Manual for the columnar layout of various types of source files.

PWB/Copy is written in DCALGOL, a variant of Unisys MCP Extended Algol. It uses the GETSTATUS API to search MCP directories and the Redirector virtual I/O handler to implement SMB/CIFS file access across a network.

Installation and Configuration

Download the ZIP file, which contains the source for PWB/Copy and a couple of example WFL jobs showing its use. These files are in MCP PWB (Programmers Workbench) format. The decompressed source can be uploaded to an MCP environment using the PWB editor, the MCP File Copier utility, the Windows Explorer extension for MCP Client Access Services, the MCP's SYSTEM/NXSERVICES/PCDRIVER utility program, or FTP.

Compile the PWB/Copy program with DCALGOL. If your site has not licensed Algol (which includes DCALGOL), you can compile PWB/Copy with DMALGOL instead. No special compiler options or run-time task attribute settings need be compiled into the codefile.

Running PWB/Copy

PWB/Copy has no parameters or command syntax. It is configured entirely through the use of file and task attributes. The following example shows the basic idea:

    RUN OBJECT/UTIL/PWB/COPY;
        SW1=TRUE;       % IGNORE BYTE-STREAM FILES
        FILE DISK=SRCE/UTIL ON LIBPACK;
        FILE SHARE (FILENAME=TEST/UTIL,
            IOHSTRING="SERVER=WINSRV SHARENAME=TESTSHARE TIMEOUT=5");

PWB/Copy has three files to which attributes can be equated externally:

• DISK: Identifies the MCP directory to transfer.
• SHARE: Redirector file for the remote share.
• LINE: Printer file.

It also has two task attributes that affect its behavior:

• SW1: ignore byte-stream and word-stream files.
• SW2: apply file name extensions unconditionally.

The file DISK specifies an MCP file or directory of files that is to be copied to the SMB/CIFS share. This logical file is also used to read the physical MCP files as they are being transferred. The user should not assign any attributes of this file, other than FILENAME, LFILENAME, TITLE, LTITLE, or FAMILYNAME. If no FILENAME is specified, all files under the user's directory will be examined for transfer (based on the special standardform name 48"030100"). To specify all files under a given usercode (e.g., USAH), use the alternate form for indicating a usercode in a file name, thus:

    FILE DISK=*USERCODE/USAH ON PROGPK;

Only one MCP directory of files can be converted and transferred in one execution of PWB/Copy. All subdirectories of the specified MCP directory will be included in the transfer, however.

The file SHARE specifies the SMB/CIFS destination. It is internally declared by default with the attribute REDIRECTION=TRUE, which implies KIND=VIRTUAL, IOHLIBACCESS=BYFUNCTION, and IOHFUNCTIONNAME="REDIRSUPPORT". It is also declared to translate the MCP source text to ASCII in the converted files.

Normally, the IOHSTRING attribute for this file should specify the server and SMB/CIFS share name, with the FILENAME or LFILENAME attribute specifying the directory under the share where the files will be transferred. Alternatively, a pseudo-UNC form of FILENAME attribute can be used to specify the server and share, e.g.,

    FILE SHARE (FILENAME=*UNC/SERVERNAME/SHARENAME/DIR1/DIR2,
        IOHSTRING=" ... ");

Other options exist for specifying the share name and IOHSTRING parameters. See the Redirector documentation in the I/O Subsystem Reference Manual for details on the IOHSTRING attribute, file titles, user credentials, and connecting to remote SMB/CIFS shares.

If no FILENAME attribute is defined for the SHARE file, converted files will be stored starting at the root folder of the share. No attributes other than IOHSTRING and one of the FILENAME variants should be equated for this file.

Only FILEKINDs that have a corresponding PWB extension defined, plus MCP printer-backup (spool) files and a few others, will be converted and transferred. All other files in the source directory will be ignored.

• Record-oriented files will be formatted according to PWB conventions concerning record size and trailing spaces. Source files (e.g., FILEKIND=ALGOLSYMBOL, COBOL85SYMBOL) will be transferred with sequence number fields, patchmark fields, and space padding as required for PWB Windows-file formats.
• Printer-backup files will be formatted as line-delimited files with CR/LF/FF carriage control.
• Byte-stream files will be transferred as is unless the task attribute SW1=TRUE. When this attribute is set, files with FILECLASS other than RECORDORIENTED will be ignored. SW1=FALSE by default.
• Empty files (those with LASTRECORD=-1) will not be transferred.

All text is written to the remote file in ASCII. Thus, this program is not recommended for use with EBCDIC data files containing packed-decimal or binary data, since character translation will probably corrupt that data.

The node names of the MCP source file below those specified for the title of the DISK file will be preserved when transferring files to the SMB/CIFS share, except that the final node will normally have a PWB extension appended to it based on the value of the MCP file's FILEKIND attribute. This extension will not be appended to the final node, however, if all of the following are true:

• The FILECLASS of the file is not RECORDORIENTED, i.e., it is a byte-stream or word-stream file.
• The final node name contains a period ("."), i.e., it appears to have an extension already.
• The task attribute SW2 is FALSE (the default). If SW2=TRUE, extensions are applied to the final node name unconditionally.

The file LINE is by default a printer file. It records a trace of the MCP directories traversed and the results of files transferred. In most cases, this output is not very useful, so you may wish to suppress it using the following file equation:

    FILE LINE (DUMMYFILE);

The Redirector is a very chatty interface, and may display numerous messages as it opens and closes connections with the remote system. PWB/Copy must probe for the existence of directories and create them on the remote share if they do not already exist. These activities typically generate messages that may appear to be errors, but merely represent unsuccessful probes of the remote file system. A very common (and almost always benign) message from the Redirector while running PWB/Copy is:

REDIRECTOR:SMB make new directory (SMBmkdir) failure. SMB Error: ERRdos, Directory invalid

If you see the following, though, you probably have a credentials problem:

REDIRECTOR:SMB session log on (SMBsesssetupX) failure. SMB Error: ERRdos, Access denied

If you see the following, especially if preceded by a delay at least as long as the IOHSTRING timeout you have specified, the MCP Redirector cannot locate the remote host—check that you have the correct SHARENAME parameter, then try using a fully-qualified DOMAINNAME or IPADDRESS parameter in IOHSTRING:

REDIRECTOR:Device/File open failed. Timeout Opening NetBIOS Session.
REDIRECTOR:Result Descriptor=000000 000000

So, What Good Is It?

PWB/Copy is useful when you want to move large numbers of MCP source code files to a Windows or *nix environment.

Since it runs in the MCP environment, PWB/Copy can access files in subdirectories that would otherwise be hidden to SMB/CIFS clients when a file exists with the same name as the directory. It is not uncommon in MCP shops to have source file names of the following form:

    (LIB)SOURCE/APP/SOME/REPORT
    (LIB)SOURCE/APP/SOME/REPORT/SPECIALCASE
    (LIB)SOURCE/APP/SOME/REPORT/OLDVERSION

SMB-based source copy tools running from outside the MCP (such as MCP File Copier) will only see and transfer the first of these three files. PWB/Copy will find and transfer all of them.

PWB/Copy will only convert and transfer files with FILEKINDs that map to a recognized PWB file name extension or MCP printer-backup. It will skip over object codefiles, DMSII and ISAM files, etc.

PWB/Copy will preserve the subdirectory structure that it finds and automatically create subordinate directories on the remote share as necessary. Existing directories on the remote share will be preserved, unless PWB/Copy transfers a file with the same name as the remote directory (but given that the file names will all have PWB extensions, a conflict with a directory name is unlikely). Existing files on the share with the same name as one being transferred by PWB/Copy will be replaced, however (as is normally the case in the MCP).

Once the files are transferred to the remote share, what can you do with them there? The short answer is anything you want, but here are some things we've found useful:

• Editing. Since PWB format is a form of standard text file, the files generated by PWB/Copy can be opened and edited by virtually any other text editor or word processor. PWB itself can open and edit source and data files from either Windows or MCP file systems, and can save files from one to the other. Note that once you modify a file in PWB format with another editor, it probably won't be in PWB format anymore, and you will need to run the file through PWB's Import Wizard to open it with PWB again.
• PWB Patch Mode. The PWB editor can edit source files directly, or by using patch mode. In patch mode, the base source is not modified; instead, all changes are recorded in a separate patch file. All of the MCP compilers understand how to merge a file of source patches with a base source file during compilation – the merge is done according to the sequence numbers in the source and patch file records. Multiple patch files can be merged by the standard SYSTEM/PATCH utility program into a single patch file that is then fed to the compiler. When using PWB in this mode, the source and patch files can be stored on different systems. In particular, a copy of the base source can be on your local workstation or a Windows or *nix file server, and the patch file can be saved to the MCP file system. When accessing the base source from your local environment, many operations in PWB are faster than when accessing the base source from the MCP, especially search and replace (which is much faster), and particularly if the base source is a large file.
• Multi-file Searching. Many Windows and *nix text editors have the ability to quickly search whole directory trees of files. If a copy of the entire source for your application is stored in your local environment, you can search for variable names, values, and substrings of text across the entire application. Our experience is that hundreds to thousands of typical source files in a directory tree can be searched in a couple of minutes or less. This is a tremendously useful capability to have when doing conversions or during impact analysis before major enhancements.
• Archiving. PWB/Copy makes it easy to push a complete copy of your application's source code to a system where it can be archived to CD-ROM, compressed in a ZIP archive, or otherwise reliably and inexpensively preserved in a format that you can expect will remain viable for a long time.
• Versioning and SCM. There are a large number of source code management (SCM) tools, such as Subversion, Perforce, and GIT, that can maintain versioned archives of source code, along with other artifacts. Few of these tools have a direct interface with the MCP environment, but virtually all of them can be used with Windows and/or *nix systems. Using PWB/Copy to transfer a complete set of source files to an environment where your favorite SCM tool can get at them is a first step in being able to manage your MCP source within that tool.

Project Participation

We welcome others to participate in this project and submit contributions to the software and documentation. Please contact the Project Admin for information on joining the project.

If you wish to make changes to PWB/Copy, please try to preserve the existing sequence numbers in the source code lines as much as possible. Sequence numbers are a foreign concept in most development environments, but they are an important part of serious development for MCP software, so try to change them as little as possible.

References

Unisys documentation is available without cost on their support web site, http://support.unisys.com.

• The Unisys MCP Extended Algol language is documented in Algol Programming Reference Manual, Volume 1: Basic Implementation (8600 0098).
• The DCALGOL language extensions to Extended Algol are documented in the DCALGOL Programming Reference Manual (8600 0841).
• The GETSTATUS API used by PWB/Copy to search MCP file system directories and obtain file information is documented in the GETSTATUS/SETSTATUS Programming Reference Manual (8600 0346).
• The Redirector virtual I/O handler used to implement SMB/CIFS file access is documented in the I/O Subsystem Programming Guide (8600 0056).
• File attributes (including FILEKIND) are documented in the File Attributes Programming Reference Manual (8600 0064).
• For an introduction to the Redirector, you may be interested in the following presentation from the 2007 UNITE conference: http://www.digm.com/UNITE/2007.