To merge tally and joint map updates from the field

Large pipeline projects have many survey crews working on multiple sections of the pipeline at any one time. In addition, different survey crews can work on different phases and sections of the pipeline on a daily basis, collecting tally and joint mapping data or using the previously collected tally and joint mapping data. To maximize the efficiency of each survey crew, all crews should have access to the tally and joint map data collected by other field crews the previous day.

If you are syncThe contents of the Tally folder are not synchronized when you upload job data to or download data from the cloud. If you use Trimble Connect to synchronize data on the controller with the cloud, when you return to the office you must manually copy the contents of the Tally folder from the controller to a location where the office computer can access it, such as a location on your office file server.

Use the Tally and Joint Map Updater utility to merge the updated tally and joint map data from multiple field crews into a master set of files in the office at the end of each day. You can then send out the updated master files to each field crew, ready for the next day's work. An XML file containing all the merged data is also available for generating custom reports from.

To download the Tally and Joint Map Updater utility, go to https://geospatial.trimble.com/product-and-solutions/access and click Downloads / Trimble Access Pipelines.

To merge tally and joint map data

To obtain a complete set of tally and joint map files that you can use as a master set, set up the Pipelines project on one controller and use that controller to open/create the tally file and add at least one joint map record.
Transfer the set of tally and joint map files from the controller to a folder that can be accessed by the office computer. This folder now contains the master set of CSV, DFN, IDX, and MAP files.
Below the folder containing the master files, create a subfolder for each field crew that updates the tally and joint map files.
To update the files in the field crew subfolders on the office computer, copy the tally files from each controller to the appropriate field crew subfolder in your network location.
Use the Tally and Joint Map Updater utility on the office computer to merge the updated data from the field crew subfolders into the master set of files.
If required, click Edit in the Tally and Joint Map Updater utility to delete joint definitions from the master file or to rename joint IDs in the master file.

The Rename button in the Edit master tally file dialog is available only when a valid joint ID is entered into the Current joint ID field. The utility will check to ensure that the specified New joint ID does not already exist in the master tally file.
Copy the master set of files onto each controller, so that each field crew has the latest data ready to begin field work at the start of the following day.

To automate merging using a batch file

When the UpdateMasterCsvFileInstaller.exe installation program is run, it installs the updater program into the following default folder:

Program Files\Common Files\Trimble\Pipelines Update Master Csv File (on a 32-bit operating system)
Program Files (x86)\Common Files\Trimble\Pipelines Update Master Csv File (on a 64-bit operating system)

As part of the installation it creates an Update (.bat) batch file that can be used to automate the process of merging tally and joint map updates from the field into master tally and joint map files. The Update batch file that is created is designed to work with a folder structure where the crew file folders are subfolders of the folder containing the master files. Once you have created an appropriate structure you can copy the Update.bat file from the program installation folder into the folder containing the master files and then run it from there.

By default the Update batch file is designed to operate with the master CSV file name supplied to it as a command line parameter, which means that it needs to be run from a Command Prompt window (DOS Shell). However, you can edit the batch file to have the name of the master CSV file specified directly in the batch file if you wish. This allows the batch to be run by double clicking the batch file name in File Explorer. To do this:

Open the Update batch file in a suitable text editor (for example, Notepad).
Modify the fourth line in the file. This line is initially created as: set masterCsvFile=%1. Change the %1 at the end of the line to the name of the master tally CSV file, remembering to include the file name in double quotes if it includes space characters.

For example, if your master tally CSV file name is ‘My tally file.csv’ then replace the %1 characters with “My tally file.csv”. The double quotes are optional if there are no space characters in the file name. No file path is required in the specification of the master CSV file name since the Update.bat file being edited is in the same folder as the master .csv file.
Save the edited file.

When the master tally and joint map files are updated by the utility program, an updated copy of the associated XML file (used for reporting purposes) is also created alongside both the master and contributing field files (in their subfolders). This means that these XML files are available for creating reports on the computer using the ASCII File Generator program. The ASCII File Generator program can also be run in an automated fashion from a batch file. If the ASCII File Generator program has been installed on the computer prior to the UpdateMasterCsvFileInstaller.exe installation program being run, the created Update.bat batch file will include some commented out report commands to show how reports can be automatically produced as part of the data merging process.

For more information on the ASCII File Generator, refer to the topic Custom import & export formats in the Trimble Access Help.

File handling

The tally file and its associated files are stored in the Trimble Data\Common\Tally folder on the controller.

The Tally folder contains the following files:

File	Type	Contains	Created when
Tally	.csv	Joint attribute data	One of the following occurs: A CSV file (such as the pipe manufacturer manifest) copied to the controller is opened in the Check tally or Create tally screen. A new file is created in the Create tally screen.
Manifest	.csv	Joint attribute data	A CSV file from the pipe manufacturer is copied to the controller for checking against data entered in the Create tally screen.
Definition	.dfn	Configured tally options	Tally options are configured in the Pipeline options screen.
Master index	.idx	Unique ID, status flags, and timestamp and position information for each joint	Tally options are configured in the Pipeline options screen.
Joint map	.map	Joint map records, including timestamp and position information	The first time a joint map record is added to the pipeline project.
Report	.xml	Data from the other files in the folder	Files are updated using the Tally and Joint Map Updater utility. The .xml file is used for creating reports on the office computer.

The .idx file may contain any of the following flags:

0 – The joint has not been checked or modified

1 – The joint has been checked

2 – Existing attribute values for the joint have been modified

3 – The joint has been checked and modified

Position information in the .idx and .map file is provided as Lat, Long, Height values, then Northing, Easting, Elevation values, followed by the Station value.

When merging updated data from the field, the Tally and Joint Map Updater utility searches for matching unique tally ID values, so it can handle files where there are extra or missing, lines. It can also handle the situation where a unique tally ID has been entered for an existing master file entry that previously had no unique tally ID (as long as no other changes have been made to the line). Parallel updating of the master index file is also carried out to keep the indices in step with the csv file and to update any recorded positions. If joint mapping files (*.map) are available alongside the selected csv files, then the master joint mapping file is updated using any new/updated details available in the new joint mapping file. A log of all the changes made to the master files is recorded in a log file stored alongside the master files, and it can be viewed at the completion of the update process.