spark command line tool. In current versions of AMPS, spark is included in the AMPS distribution.
The instructions in this FAQ are useful for a one-time transfer, for example, loading sample data into a QA environment for testing. For production use, AMPS replication is the recommended approach for low-latency delivery of messages from instance to instance as described in the AMPS User's Guide.
These instructions assume that your data does not contain embedded newlines: if your data does contain newlines, we recommend writing a simple program that simply does a SOW query on one instance and publishes the results to the other instance or using the -file option in more recent versions of spark.
Copy from a SOW File
To recreate a SOW from a server that is no longer running, use
amps_sow_dump to extract the records from the SOW and use spark to publish those records into the destination AMPS instance.
This process assumes that the records do not contain embedded newlines, since
amps_sow_dump uses newline characters to delimit messages.
$ amps_sow_dump /path/to/sow/file.sow > records.txt $ spark publish –server new_server –topic new_topic –type message_type –file records.txt
Copying from a SOW topic in a running instance
To recreate a SOW on a server that is currently running, use the
spark sow command to retrieve records from the source topic, and use the
spark publish command to publish records to the destination topic.
$ spark sow -server old_server topic old_topic -type message_type \
| spark publish –server new_server –topic new_topic –type message_type
Notice that, because this is an operation on a running instance of AMPS, any updates that are made to the SOW topic after the
sow command runs will not be published to the new topic.
If the messages are binary or contain embedded delimiter characters, and the number of records can be easily stored on the system where the command is being run, you can use the
-file flag to store messages to a compressed file, and then republish the messages from that file.
$ spark sow -server old_server topic old_topic -type message_type -file messages.zip
$ spark publish –server new_server –topic new_topic –type message_type -file messages.zip
Keywords: sow synchronization, move sow, copy sow, synchronize sow, move messages