Data Replication: Customer Responsibilities for Maintenance
Bullhorn Data Replication relies on a partnership between Bullhorn and your organization. Bullhorn maintains the core Data Replication service and the data synchronization logic. Your organization is responsible for the operational health of your Data Replication installation, including keeping your database in good working order and ensuring the three client-side Data Replication Java applications are properly deployed, monitored, and kept current with the latest releases.
This article outlines the specific responsibilities your Database Administrators (DBAs) and System Administrators (SysAdmins) must fulfill to keep Bullhorn Data Replication running without interruption. For an overview of Bullhorn Data Replication and its hardware and software requirements, see Understanding Bullhorn Data Replication.
Summary of Key Responsibilities
The table below summarizes the operational responsibilities your organization's Database Administrators and System Administrators must fulfill to maintain Bullhorn Data Replication. Full detail on each responsibility is provided in the sections that follow this table.
| Role Responsible for Bullhorn Data Replication Maintenance | Bullhorn Data Replication Responsibilities |
|---|---|
| Database Administrator (DBA) |
|
| System Administrator (SysAdmin) |
|
Database Administrator Responsibilities for Bullhorn Data Replication
Your organization's Database Administrators are responsible for the operational health of the target database where Bullhorn Data Replication writes synchronized data. The tasks in this section are standard database maintenance tasks required for any mission-critical production database, applied specifically to your Bullhorn Data Replication target database.
Bullhorn Data Replication Database Monitoring and Health
-
Monitor disk space on the Bullhorn Data Replication target database server. Bullhorn Data Replication writes continuously. Ensure your database server has sufficient free disk space to accommodate inbound Bullhorn Data Replication data without reaching capacity thresholds. Set alert thresholds to trigger when disk usage on the Bullhorn Data Replication target database server exceeds 70-80%.
-
Monitor CPU and memory on the Bullhorn Data Replication target database server. Ensure the database server has adequate resources to process incoming Bullhorn Data Replication transactions. Sustained high CPU or memory usage on this server may indicate performance bottlenecks that require investigation.
-
Monitor database connectivity between the Bullhorn Data Replication applications and your target database. Review the Bullhorn Data Replication application logs for connection timeout errors or repeated connection failures. The Bullhorn Data Replication application logs will explicitly flag database connectivity issues. Consistent timeout errors in these logs indicate database connection problems that require investigation and troubleshooting.
-
Configure backup and recovery procedures for the Bullhorn Data Replication target database. Bullhorn Data Replication writes critical business data to your target database. Establish and test backup and disaster recovery processes for this database that align with your organization's data retention and recovery time objectives (RTO/RPO).
Bullhorn Data Replication Database Maintenance
-
Maintain indexes on the Bullhorn Data Replication target database based on query patterns. Create and maintain indexes on the tables where Bullhorn Data Replication writes data, based on how your organization intends to consume that data (through reports, direct queries, or data warehouse pipelines). Regular index maintenance prevents query performance degradation on the Bullhorn Data Replication target database. Perform index rebuilds or defragmentation according to SQL Server best practices.
-
Keep statistics current on the Bullhorn Data Replication target database. Keep table and index statistics updated on the Bullhorn Data Replication target database so the SQL Server query optimizer can generate efficient execution plans for Bullhorn Data Replication writes. Balance statistics maintenance with write performance: excessive indexing and overly aggressive statistics updates can slow down Bullhorn Data Replication writes. Work with your indexing strategy to avoid overcomplicating the index structure on the Bullhorn Data Replication target database.
-
Monitor and manage growth of the Bullhorn Data Replication target database. Track the rate at which tables in the Bullhorn Data Replication target database grow due to incoming Bullhorn Data Replication data. Plan for storage capacity on this database accordingly and implement data retention or archival policies if needed.
-
Apply database patches and security updates to the server hosting the Bullhorn Data Replication target database. Keep your SQL Server instance current with vendor-released patches and security updates. This ensures security, stability, and compatibility with Bullhorn's data synchronization processes.
Bullhorn Data Replication Database Monitoring and Alerting
Configure database alerts for the Bullhorn Data Replication target database that cover the following critical metrics: available disk space, failed backup jobs, connection pool exhaustion, and transaction log growth.
System Administrator Responsibilities for Bullhorn Data Replication
Your organization's System Administrators are responsible for ensuring the three Bullhorn Data Replication Java applications remain deployed, operational, and current on your Windows servers.
The Three Bullhorn Data Replication Applications
Bullhorn Data Replication relies on three Java applications that run on Windows. All three applications must be running for Bullhorn Data Replication to function, and all three communicate with Bullhorn's Data Sync Services. The table below describes the purpose and deployment location of each Bullhorn Data Replication application.
| Bullhorn Data Replication Application | Purpose of This Bullhorn Data Replication Application | Where This Bullhorn Data Replication Application Runs |
|---|---|---|
| Manager | Orchestrates Bullhorn Data Replication synchronization jobs and manages the Bullhorn Data Replication subscription connection to Bullhorn's Data Sync Services. | Runs on your Bullhorn Data Replication application server(s). |
| Launcher | Triggered by the Bullhorn Data Replication Manager application to execute scheduled synchronization jobs and to update the Bullhorn Data Replication Client application. | Runs on the same server as the Bullhorn Data Replication Manager application. |
| Client (client-ems.jar) | Performs the actual Bullhorn Data Replication data extraction and loads data into your target database. | Runs on the same server as the Bullhorn Data Replication Manager and Launcher applications. |
Bullhorn Data Replication Application Deployment and Execution
-
Verify the Bullhorn Data Replication Windows scheduled task is configured. Bullhorn Data Replication is controlled by a single Windows scheduled task that runs the Bullhorn Data Replication Manager application. Confirm the following about this scheduled task:
- The Manager application's scheduled task is created and enabled.
- The scheduled task is configured with the correct service account credentials and has appropriate file system and database permissions for Bullhorn Data Replication.
- The scheduled task is configured to automatically restart the Bullhorn Data Replication Manager application if it fails or becomes unresponsive.
- The scheduled task is configured to restart whenever your Windows server reboots, ensuring Bullhorn Data Replication resumes automatically after maintenance or unplanned downtime.
-
Verify Bullhorn Data Replication application configuration. Confirm the following:
- All three Bullhorn Data Replication JAR files (Manager, Launcher, and Client) are deployed in the correct directory.
- The Bullhorn Data Replication application.properties configuration file contains the correct settings for your Bullhorn Data Replication subscription and target database.
Bullhorn Data Replication Application Monitoring
-
Monitor for Bullhorn Data Replication application failures. The Bullhorn Data Replication Manager and Client applications run continuously in the background and generate their own log files. Implement proactive monitoring to detect Bullhorn Data Replication application failures:
- Use Windows Event Viewer to monitor scheduled task execution history for Bullhorn Data Replication Manager application failures.
- Configure alerting (email, SMS, or ticketing system integration) to notify your operations team immediately when the Bullhorn Data Replication Manager application's scheduled task fails to execute or exits unexpectedly.
- Monitor the Bullhorn Data Replication Client application logs for persistent ERROR entries, which may indicate a separate failure requiring investigation.
-
Monitor Bullhorn Data Replication application logs. The Bullhorn Data Replication applications write detailed logs. Review these Bullhorn Data Replication application logs regularly for the following entry types:
- ERROR log entries indicate Bullhorn Data Replication failures that require immediate investigation, such as connection errors, authentication failures, or database write failures.
- WARN log entries indicate non-critical Bullhorn Data Replication issues that may resolve themselves or may indicate emerging problems worth monitoring.
- INFO log entries provide context on successful Bullhorn Data Replication operations and can help correlate the timing of issues.
Check the Bullhorn Data Replication application logs after any system changes, database maintenance, or network configuration updates.
-
Monitor Bullhorn Data Replication connectivity to Data Sync Services. Verify that the Bullhorn Data Replication Manager and Client applications maintain successful connections to Bullhorn's Data Sync Services. Connection errors in the Bullhorn Data Replication application logs indicate network configuration issues, firewall problems, or service availability issues that require investigation.
Bullhorn Data Replication Application Restart and Recovery
The Bullhorn Data Replication Manager application runs continuously and handles most operations internally, including keeping the Launcher and Client applications running and healthy, monitoring for synchronization job completion, handling automatic application restarts if an internal process fails, and updating the Client JAR file if self-updating is enabled.
Your System Administrators are responsible for the following Bullhorn Data Replication restart and recovery tasks:
-
Monitor Bullhorn Data Replication task execution. If the Bullhorn Data Replication Windows scheduled task fails, the Manager application has exited for a critical reason. This requires immediate investigation:
- Review recent Bullhorn Data Replication application logs for ERROR entries indicating why the Manager application exited.
- Check for database connectivity issues, disk space problems, or configuration errors affecting Bullhorn Data Replication.
- Once the underlying issue is resolved, manually restart the Bullhorn Data Replication scheduled task or wait for the next scheduled execution time.
-
Escalate critical Bullhorn Data Replication failures to Bullhorn Support. If the Bullhorn Data Replication Manager application fails to start or crashes repeatedly (more than once per day), contact Bullhorn Support before attempting further troubleshooting. Provide Bullhorn Support with the following:
- Recent Bullhorn Data Replication application log files (the previous 24 hours of logs).
- Windows Event Viewer entries for the Bullhorn Data Replication scheduled task.
- Any recent changes to network, database, or server configuration affecting Bullhorn Data Replication.
Bullhorn Data Replication Version Management and Software Updates
Automatic Updates for Bullhorn Data Replication Applications
The Bullhorn Data Replication applications are kept current through an automatic update mechanism: the Bullhorn Data Replication Manager application updates the Launcher application, and the Launcher application updates the Client application (client-ems.jar). This chain of updates ensures your Bullhorn Data Replication installation stays current with critical bug fixes, performance improvements, and feature updates without requiring manual intervention from your team.
By default, auto-updating is enabled for Bullhorn Data Replication. The Manager and Launcher applications will automatically check for and apply updates when they run.
Bullhorn Data Replication Update Options
Your organization can manage Bullhorn Data Replication version updates using one of the following two options.
Option 1: Enable Self-Updating for Bullhorn Data Replication (Recommended)
What you do: Nothing. Leave the auto-update feature enabled in your Bullhorn Data Replication application.properties configuration.
What happens: Your Bullhorn Data Replication applications automatically stay current. When the Manager application runs, it checks for and applies Launcher application updates. When the Launcher application runs, it checks for and applies Client application updates. Bullhorn releases Bullhorn Data Replication updates regularly, and your installation receives them automatically within 1-2 synchronization cycles. You may see brief log entries noting that a Bullhorn Data Replication update was applied, but synchronization operations continue uninterrupted.
Support status: Fully supported. Your team is on the current Bullhorn Data Replication version and entitled to all Bullhorn support.
Your responsibilities: Continue the Bullhorn Data Replication monitoring and maintenance tasks outlined earlier in this article. Your team does not need to manually manage Bullhorn Data Replication version updates.
Option 2: Disable Self-Updating and Manually Manage Bullhorn Data Replication Versions
What you do: Opt out of the auto-update feature in your Bullhorn Data Replication configuration and assume responsibility for monitoring and deploying new Bullhorn Data Replication application versions.
What happens: Your Bullhorn Data Replication applications do not update automatically. Your team must monitor Bullhorn release announcements and documentation for new Bullhorn Data Replication versions, download new versions from Bullhorn's customer portal, deploy the new Bullhorn Data Replication JAR files to your server (Manager, Launcher, and/or Client as needed), and restart the Bullhorn Data Replication Manager application for the updates to take effect.
Support approach: Bullhorn Support will prioritize assisting you with Bullhorn Data Replication upgrades. However, if you report a Bullhorn Data Replication issue and your version is not the latest released version, the first step will be to upgrade Bullhorn Data Replication before troubleshooting. This ensures you are not affected by known issues already fixed in newer Bullhorn Data Replication releases.
Your responsibilities: Your team owns the entire Bullhorn Data Replication version management lifecycle and must ensure timely updates are deployed to maintain Bullhorn Data Replication service continuity.
Staying Current with Bullhorn Data Replication Versions
Whether your organization uses auto-update or manual updates for Bullhorn Data Replication, it is important to stay on current versions of all three Bullhorn Data Replication applications. Versions of the Bullhorn Data Replication applications that fall significantly behind the latest release may have compatibility issues with Bullhorn's Data Sync Services. To ensure the best Bullhorn Data Replication support experience:
- If auto-update is enabled for Bullhorn Data Replication, all three Bullhorn Data Replication applications are automatically kept current and no action is needed from your team.
- If auto-update is disabled for Bullhorn Data Replication, establish a process to review and deploy new Bullhorn Data Replication versions regularly (monthly or per your IT change management schedule).