Giter Club home page Giter Club logo

bucardo's Introduction

Bucardo - a table-based replication system

DESCRIPTION:
------------

This is version 5.6.0 of Bucardo.

COPYRIGHT:
----------

    Copyright (c) 2005-2023 Greg Sabino Mullane <[email protected]>

REQUIREMENTS:
-------------

    build, test, and install Perl 5                (at least 5.8.3)
    build, test, and install PostgreSQL            (at least 8.2)
    build, test, and install the DBI module        (at least 1.51)
    build, test, and install the DBD::Pg module    (at least 2.0.0)
    build, test, and install the DBIx::Safe module (at least 1.2.4)
    
    You must have at least one database that has PL/pgSQL and PL/Perl installed.
    Target databases may need PL/pgSQL.


INSTALLATION:
-------------

To install this module type the following:

   perl Makefile.PL
   make
   make test (but see below first)
   make install

EXAMPLES:
---------

See the test suite in the t/ subdirectory for some examples.

WEBSITE:
--------

Please visit https://bucardo.org for complete documentation.

DEVELOPMENT:
------------

To follow or participate in the development of Bucardo, use:

git clone [email protected]:bucardo/bucardo.git

GETTING HELP:
-------------

For general questions and troubleshooting, please use the [email protected]
mailing list.  GitHub issues which are support-oriented will be closed and referred to
the mailing list anyway, so help save time for everyone by posting there directly.

Post, subscribe, and see previous archives here:

https://bucardo.org/mailman/listinfo/bucardo-general

bucardo's People

Contributors

alianobii avatar aolmezov avatar burbon avatar df7cb avatar gustavotonini avatar hasegeli avatar jarec avatar jeltz avatar jonjensen avatar mat813 avatar matlads avatar mnp avatar mwpastore avatar redsysman avatar rlafferty avatar rosser avatar theory avatar turnstep avatar wiml avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

bucardo's Issues

Bucardo_ctl reports date in the future for 'Last good time'

ID: 56
Component: bucardo_ctl
Version: 4.4.0
Date: 2010-11-29 10:09 EST
Author: geoff ([email protected])


Hi,

Bucardo_ctl is reporting dates in the future for the 'last good time' field
when I do a bucardo_ctl status syncname. The server I'm running this on is
based in Tasmania, Australia where the current timezone offset is currently
UTC/GMT +11 hours.

Steps to reproduce:

Just run bucardo_ctl status syncname when your system timezone is set to
Australia/Hobart. This is on a centos 5.5 64 bit box.

Expected behaviour:

Last good time should be in the past

Actual behaviour:

bucardo_ctl status syncname is currently reporting:

Last good time: Nov 30, 2010 04:09:55
(and Last bad time: Nov 30, 2010 04:09:29)

But the system date command reports the current date as:

Mon Nov 29 20:53:30 EST 2010

And psql reports the correct time:

bucardo=# select to_char(current_timestamp, 'YYYY-mm-dd HH24:MI:SS TZ');
         to_char
 2010-11-29 21:07:46 EST
(1 row)

Environment

OS:
Centos 5.5 i86_64

Packages:
bucardo-4.4.0-2.el5
perl-DBIx-Safe-1.2.5-6.el5
perl-version-0.7203-1.el5
nagios-plugins-perl-1.4.15-2.el5
perl-5.8.8-32.el5_5.2
perl-DBD-Pg-2.17.1-1.el5.rf
perl-DBI-1.52-2.el5
compat-postgresql-libs-4-1PGDG.rhel5
postgresql-libs-8.4.5-1PGDG.rhel5
postgresql-devel-8.4.5-1PGDG.rhel5
postgresql-libs-8.1.21-1.el5_5.1
postgis-1.3.6-1.rhel5
postgresql-8.4.5-1PGDG.rhel5
postgresql-contrib-8.4.5-1PGDG.rhel5
postgresql-server-8.4.5-1PGDG.rhel5
postgresql-plperl-8.4.5-1PGDG.rhel5

strict_checking Not Documented

The strict_checking parameter is not documented. I found that I needed it for a sync, and not only is it not documented, but it doesn't work for add sync, either. It does work for update sync, however.

It should probably be documented and supported for both add and update commands for both syncs and tables.

Need to fix the password option for MD5 passwords

In this thread, Sean M. Pappalardo reports a number of issues with MD5 passwords:

FIrst, the install command never prompts for a password.

Then there seem to be some issues with MD5 passwords:

I have to specify
-dbpass= on every invocation of bucardo_ctl or I get
an error:

DBI connect('dbname=bucardo','bucardo',...) failed: fe_sendauth: no
password supplied at /usr/local/bin/bucardo_ctl line 194

(I'm using md5 auth for the bucardo user.)

@turnstep, do have some pointers as to what, exactly, should be changed to fix this issue?

Kick hangs after reload config

I have a multi-master configuration running on 4.99.6. Working fine, but then try this:

bucardo update sync mysync autokick=0
bucardo reload config

This properly disables autokick. However, a manual kick just hangs:

bucardo kick mysync 0
Kick mysync: [48 s] ^C

(That ^C is me killing it.) Turning autokick back on does not help:

bucardo update sync mysync autokick=1
bucardo reload config
bucardo kick mysync 0
Kick mysync: [14 s] ^C

Data is not being copied over, either, even if I insert some. That is, autokick does not work. In fact, reload hangs, too:

bucardo reload mysync
Reloading sync mysync...^C

The only thing I can do to get it all working again is restart:

$ sudo service bucardo restart
Stopping bucardo service:                                  [  OK  ]
Starting bucardo service:                                  [  OK  ]

No autokick works and kick no longer hangs:

bucardo kick references_sync 0
Kick references_sync: [1 s] DONE!

reload no longer hangs, either:

bucardo reload references_sync
Reloading sync references_sync...

Perhaps reload config is not actually doing everything it should?

Need more control over the reason file(s)

The reason file gets opened in the current directory. This is a bit annoying for init scripts. And in any event one might not want such a file.

I propose two changes:

  • Add a new option to disable logging to the reason file(s). Or perhaps don't log to it at all if there is no reason?
  • Have the reason file live in --debugdir instead of whatever the current directory is.

Also, maybe we want to change its name from bucardo.restart.reason.log, so that we then don't also end up with bucardo.restart.reason.log.log? I think I would just make it bucardo.restart.reason and then the appending file would be bucardo.restart.reason.log.

Delta relation doesn't exist

I made some database schema changes, but Bucardo didn't really like this. So I did a cascade drop of the Bucardo schema database and dropped bucardo triggers from all synced tables. Then I re-installed Bucardo. I still get the same error as before, and can't seem to get rid of it. Syncing is still working fine, so it may be something small.

(20693) [Thu Feb  7 16:50:11 2013] VAC Warning! VAC was killed at line 6773: DBD::Pg::st pg_result failed: ERROR:  relation "bucardo.delta_16693" does not exist
LINE 1: DELETE FROM bucardo."delta_16693" USING (SELECT txntime AS t...
                    ^
QUERY:  DELETE FROM bucardo."delta_16693" USING (SELECT txntime AS tt FROM bucardo."track_16693" GROUP BY 1 HAVING COUNT(*) = 53) AS foo WHERE txntime = tt AND txntime < now() - interval '45 seconds'
CONTEXT:  PL/pgSQL function bucardo.bucardo_purge_delta(text,text) line 45 at EXECUTE statement
SQL statement "SELECT                  bucardo.bucardo_purge_delta($1, myrec.tablename)"
PL/pgSQL function bucardo.bucardo_purge_delta(text) line 13 at SQL statement at /usr/local/share/perl/5.10.1/Bucardo.pm line 6773.

Unescaped backslash causes infinite loop

In this list thread, Gabriel Weinberg writes:

We hit a problem query and bucardo kept trying it and got stuck in a loop. Is this because of some internal escaping bug?

KID Warning! Aborting due to exception for public.pages:? Error was DBD::Pg::db pg_result failed: ERROR: syntax error at or near "R"\nLINE 1: ...Appeals of the Philippines','A','1'),('//','','R','1'),('...\n ^ at /usr/local/lib/perl5/site_perl/5.10.1/Bucardo.pm line 8323.

Guessing it is the escaping of the backslash.

@turnstep asked for the version, which Gabriel replied with, 4.99.5.

VAC Processes do not Exit Immediately

I've noticed that the MCP, CTL, and KID processes shut down very quickly after a call to bucardo stop, but VAC processes hang around for a while. Should they be listening for a STOP message?

Rename Cutesy Terms

I'm thinking now might be a good time to replace two cutesy terms, "goat" and "herd". My suggestion:

• Change "goat" to "relation". This allows it to implicitly cover views and foreign tables, too.

• Change "herd" to "relgroup"

I find these terms much easier to decipher, as they say exactly what they are. Plus "relgroup" nicely corresponds to the existing "dbgroup" term.

I'm thinking all the docs would be updated, but the internal code and database can keep the cute names for compatibility's sake. And "goat" and "herd" would still be supported in all commands so that people's scripts and finger-memory don't stop working.

DBD::Pg::st execute failed

ID: 109
Component: bucardo_ctl
Version: HEAD
Date: 2012-08-01 04:18 EDT
Author: Yang Hua Jie ([email protected])


when I tried to use the command bucardo_ctl to add the sequences to be
synchronize, I encounter the following error.

$sudo bucardo_ctl add all sequences db=db1
DBD::Pg::st execute failed: ERROR: new row for relation "goat" violates check
constraint "standard_conflict_sequence" at /usr/bin/bucardo_ctl line 3670.

Implement `update customcode`

Its current implementation is:

sub update_customcode {

    die 'Write me';

} ## end of update_customcode

We probably ought to do something about that.

Processes do not respawn on back end termination

I have been killing Bucardo's database connections (because I was grepping the process table for bucardo, not Bucardo), and noticed that the processes don't restart themselves. Instead, the next time they are kicked or try to connect to the database, they die. From my log file:

FATAL:  terminating connection due to administrator command
FATAL:  terminating connection due to administrator command
FATAL:  terminating connection due to administrator command
FATAL:  terminating connection due to administrator command
FATAL:  terminating connection due to administrator command
(20809) [Mon Oct 29 13:56:24 2012] MCP Warning: Killed (line 795): Ping failed for main database!
(20809) [Mon Oct 29 13:56:24 2012] MCP Database problem, will respawn after a short sleep: 15
(20823) [Mon Oct 29 13:56:24 2012] KID Kid 20823 exiting at cleanup_kid. Sync "wheeler_sync" public.foo Reason: Ping failed for main database Line: 2710 
(20819) [Mon Oct 29 13:56:25 2012] CTL Warning! Controller for "wheeler_sync" was killed at line 5283: DBD::Pg::db do failed: no connection to the server at /usr/share/perl5/vendor_perl/Bucardo.pm line 5283.
(20819) [Mon Oct 29 13:56:25 2012] CTL Controller 20819 exiting at cleanup_controller. Reason: DBD::Pg::db do failed: no connection to the server at /usr/share/perl5/vendor_perl/Bucardo.pm line 5283.
FATAL:  terminating connection due to administrator command
(20815) [Mon Oct 29 13:56:25 2012] VAC VAC was killed at line 6316: MCP request
(20809) [Mon Oct 29 13:56:25 2012] MCP End of cleanup_mcp. Sys time: Mon Oct 29 13:56:25 2012. Database time: 2012-10-29 20:56:25.711099+00
FATAL:  terminating connection due to administrator command
FATAL:  terminating connection due to administrator command
(20809) [Mon Oct 29 13:56:25 2012] MCP Sleep time: 15
(20809) [Mon Oct 29 13:56:40 2012] MCP Respawn attempt: /usr/sbin/bucardo start "Attempting automatic respawn after MCP death"

Glad to see it try to respawn, but there is nothing in the process table.

MCP Exits when no syncs

From Bucardo.pm:

if (!$active_syncs) {
    $self->glog('No active syncs were found, so we are exiting', LOG_WARN);
    $self->db_notify($masterdbh, 'nosyncs', 1);
    $self->cleanup_mcp('No active syncs');
    exit 1;
}

This isn't too friendly to a start script. I would like to add an option to have it not exit when there are no syncs. That way we know it's running, even if it's not doing anything, and can just run restart to get it to pick up the new configs. Any reason not to?

4.99.7 requires at least Postgres 8.4

Commit c5fcf49 added the use of the array_agg function which was introduced in PostgreSQL 8.4.

I would suggest either bumping the requirements in the README or changing the code to remove the use of that function.

Does bucardo support quoted columns/tables? I have a bug in Bucardo 4.99.3

ID: 103
Component: General
Version: HEAD
Date: 2012-05-19 13:20 EDT
Author: Ross ([email protected])


SQL statement "SELECT validate_sync('SyncAll')"
PL/Perl function "validate_sync"
Failed to add sync: DBD::Pg::st execute failed: ERROR:  DBD::Pg::st execute
failed: ERROR:  relation "public.photographers" does not exist at line 992. at
line 30.
CONTEXT:  PL/Perl function "validate_sync" at
/usr/local/ActivePerl-5.14/site/bin/bucardo line 4157.

In fact table public."Photographers" exists, but in Postgres
"Photographers" and photographers (without quotes) are 2 different things.
The same concerns columns.

Do you support quoted colums/tables in Postgres, so it's just a small bug, or
this is not supported from the beginning?

Thanks,
Ross

Do not expect a row count from $sth->execute

Frank NJ. reported via the list that he was getting this error when trying to add a sync:

NOTICE:  Dropping older version of bucardo_sequences, then recreating empty

CONTEXT:  SQL statement "SELECT validate_sync('xxxx_sync')"
NOTICE:  Issuing rollback() due to DESTROY without explicit disconnect() of DBD::Pg::db handle dbname=wms;host=xx.xx.xx.xx;port=5432 at line 808.

CONTEXT:  SQL statement "SELECT validate_sync('xxxx_sync')"
Failed to add sync: DBD::Pg::st execute failed: ERROR:  error from Perl function "validate_sync": error from Perl function "validate_sync": DBD::Pg::db do failed: ERROR:  table "bucardo_sequences" does not exist at line 808. at line 30. at /usr/local/bin/bucardo line 3865.

Turns out the code in in validate_sync() is doing this:

        $sth = $sth{haztable};
        $count = $sth->execute('bucardo', 'bucardo_sequences');
        $sth->finish();
        if ($count < 1) { ... }

Oy. execute always returns true on success, not the number of rows. I suspect there are a bunch of places this needs to be fixed.

Not replicating, cryptic error(?) message

I'm using 4.99.6. I have two databases set up for replication and everything seems ok. When I run the replication I don't see any rows moving accross and I see this repeatedly in the bucardo log:

(2767) [Thu Jan 31 01:10:36 2013] KID Kid 2767 exiting at cleanup_kid. Sync "db_production" public.delayed_jobs Reason: Bucardo=HASH(0x112b3e8) Line: 2632

Is this actually an error? If so how do I find out what's causing it?

[Version 4.4.8] Column with name "tablename"

ID: 105
Component: bucardo_ctl
Version: 4.4.8
Date: 2012-06-27 10:38 EDT
Author: Hermann D. Schimpf ([email protected])


Hi,
I'm using Bucardo from 2 years ago. I want to congratulate for this tool, it
helped me very much.

Ok, I tell you my problem. I found a bug using Bucardo. If I have a table with
a column named "tablename", it conflicts with a internal variable from Bucardo
process (I guess) and I can't replicate this table with Bucardo. I don't know
Perl very much, I'm Java programmer. So I can't found exactly the location of
this problem on Bucardo sources.

I'll be very grateful to the fix of this bug, because I have 5 servers in
master-master replication and I need replicate that table.

Thanx

Have MCP Ignore Grandkid's Own Notices

From this discussion.

Right now the kid ignores its own messages, but ideally we would train the MCP to ignore notices from a grandkid that target itself. This can happen, for example, when autokick messages are sent when makedelta is enabled on a table. We could probably simply have the KID send the MCP a quick NOTIFY on startup.

We could also go the other way, and have the information about what PID did the kick bubble down to the CTL and then the KID, but that would be trickier.

Implement `update relgroup`

Its current implementation is:

sub update_herd {

    ## Update one or more herds
    ## Arguments: none, parses nouns
    ## Returns: never, exits

} ## end of update_herd

So either change it or, if nothing can be updated, remove it.

bucardo 5

Bucardo 4 has type sync (swap, Pushdelta ...etc), how about 5 version?

Implement `remove sequence`

The implementation currently looks like this:

sub remove_sequence {

    die 'Write me';

} ## end of remove_sequence

So it could use a little love.

Need Tests to Reliably Test Serialization failures

We might be able to fake them up using custom exception codes from a PL/pgSQL function on PostgreSQL 8.4 or higher:

david=# \set VERBOSITY verbose
david=# DO $$BEGIN RAISE EXCEPTION 'Serialization error'
       USING ERRCODE = 'serialization_failure'; END $$;
ERROR:  40001: Serialization error
LOCATION:  exec_stmt_raise, pl_exec.c:2840

unique indexes w/ NULLs cause errors and log pollution

ID: 77
Component: bucardo_ctl
Version: 4.4.7
Date: 2011-06-09 21:44 EDT
Author: Armin B. Resch ([email protected])


Check whether this fix on 4.4.4/Bucardo.pm can be applied upstream:

--- Bucardo.pm  2011-06-09 13:53:06.000000000 -0500
+++ Bucardo.pm.org      2011-05-14 22:12:30.000000000 -0500
@@ -4825,7 +4825,7 @@
                     ## Build a list of all PK values
                     my $pkvals = '';
                     for my $row (@$info) {
-                        my $inner = join ',' => map { defined ? { s/\'/''/go; s{\\}{\\\\}go; qq{'$_'}; } : 'null' } @$row;
+                        my $inner = join ',' => map { s/\'/''/go; s{\\}{\\\\}go; qq{'$_'}; } @$row;
                          $pkvals .= $g->{pkcols} > 1 ? "($inner)," : "$inner,";
                          }
                          chop $pkvals;
@@ -4876,7 +4876,7 @@
                                 my $dcount = 0;
                                 my $delcount = 0;
                                 for my $row (@$info) {
-                                    my $inner = join ',' => map { defined ? {
s/\'/''/go; s{\\}{\\\\}go; qq{'$_'}; } : 'null' } @$row;
+                                    my $inner = join ',' => map { s/\'/''/go;
s{\\}{\\\\}go; qq{'$_'}; } @$row;
                                     ## Put this group of pks into a temporary
array
                                     $delchunks[$delcount] .= $g->{pkcols} > 1
? "($inner)," : "$inner,";
                                     ## Once we reach out limit, start
appending to the next bit of the array

Table and index names not properly quoted

In this post, Sean M. Pappalardo reports:

I'm a new Bucardo user and my databases have table names with capital
letters, accents, dashes and spaces (because they're used in the UI.)
The attached patch was necessary to allow Bucardo 4.4.7 to handle these
names when I added a sync.

And the patch:

--- bucardo.schema.old  2011-11-08 22:44:05.000000000 +0100
+++ bucardo.schema  2011-11-10 23:48:30.000000000 +0100
@@ -1482,7 +1482,7 @@
         my $index5 = "bucardo_delta_txntime";
         my $safeindex1 = $index1;
         if ($safeindex1 =~ s/"/""/g or $safeindex1 =~ / /) {
-            $safeindex1 = qq{"$safeindex1"};
+            $safeindex1 = qq{$safeindex1};
         }

         if (!$found{bucardo_delta}) {

However, I don't see this code anywhere in bucardo.schema anymore. @turnstep, is this a problem for 4.x and not 5.x?

bucardo_ctl install latest master

Connecting to database 'bucardo' as user 'bucardo'
DBD::Pg::st execute failed: ERROR: column "value" of relation "bucardo_config" does not exist
LINE 1: UPDATE bucardo.bucardo_config SET value = $1 WHERE setting =...
^ at /usr/local/bin/bucardo_ctl line 5560, <> line 3.

Undefined database handle in `add all tables`

Hi

I get the following error when trying to use the bucardo_ctl commadn to add tables:

ubuntu@ip-10-171-65-39:~/Bucardo-4.5.0$ bucardo_ctl add all tables db=production
Can't call method "prepare" on an undefined value at /home/ubuntu/perl5/bin/bucardo_ctl line 3622.

This happens if I install from source or use the debian package via apt-get

Postgres server killing client DBI connection during validate_sync

ID: 99
Component: General
Version: HEAD
Date: 2012-02-23 21:24 EST
Author: Seth ([email protected])


The call to DBI->connect is failing; postgres server is killing the connection
because:

FATAL:  invalid frontend message type 87

(98754) [Thu Feb 23 20:17:27 2012] MCP Running validate_sync on "some_sync"
(98754) [Thu Feb 23 20:17:27 2012] MCP Connecting to database "some_db1"
(target)
(98754) [Thu Feb 23 20:17:27 2012] MCP DB "some_db1" Local epoch:
1330049847.14104  DB epoch: 1330049847.14107
(98754) [Thu Feb 23 20:17:27 2012] MCP DB "some_db1" Local time: Thu Feb 23
20:17:27 2012  DB time: 2012-02-23 20:17:27.141071-06
(98754) [Thu Feb 23 20:17:27 2012] MCP DB "some_db1" Local timezone: CST
(-0500)  DB timezone: US/Central
(98754) [Thu Feb 23 20:17:27 2012] MCP DB "some_db1" Postgres version: 90102
(98754) [Thu Feb 23 20:17:27 2012] MCP DB "some_db1" Database port: 5432
(98754) [Thu Feb 23 20:17:27 2012] MCP Connecting to database "some_db2"
(source)
FATAL:  invalid frontend message type 87
DBD::Pg::st execute failed: no connection to the server at
/opt/local/lib/perl5/site_perl/5.12.3/Bucardo.pm line 4849.

I was able to make the error go away by adding this around the DBI->connect
call in connect_database:

$self->glog("BEFORE DBI->connect");
$dbh = DBI->connect ...
$self->glog("AFTER DBI->connect");

Somehow adding these logs makes the error go away.

No standard_conflict Parameter to Sync Commands

Greg tells me via the mail list that in Bucardo 5, conflict resolution is set only on the sync level, not on the table level. Yet the standard_conflict parameter is not currently supported by the sync commands:

$ bucardo add sync 'foo' standard_conflict=latest
Unknown option 'standard_conflict'

So I think we should do three things:

  • Remove the standard_conflict param from the table commands
  • Add it to the sync commands
  • Rename it to conflict_method or conflict_handling_method. Not sure if the column should be renamed, too; probably not, eh?

bucardo stop Fails

Was doing some testing on a server and noticed that bucardo stop properly writes a fullstopbucardo file, but nothing happens as far as I can tell. I still see a bunch of connections in my process table.

Originally reported here. Reply from Greg here.

I will look into this a bit more to see if maybe it was just something funky on that system. I need to make sure it works properly when called by the linux start script.

Better installation suggestions

ID: 83
Component: Installation
Version: unspecified
Date: 2011-08-01 15:11 EDT
Author: Greg Sabino Mullane ([email protected])


Would be nice if the INSTALL file and/or the installer made an effort to find
and point out missing prerequisites, including how to install on common
platforms (e.g. sudo yum install perl-DBIx-Safe)

Charset issue when name of primary key contains german umlauts

I have a setup with bucardo 4.4.8 and postgres 9.1

Some primary keys used in that database contain german umlauts in their name.
It looks like bucardo has some problems writing correct triggers when these tables get added.
The problem exists only with primary keys so far, bucardo has no problem with table names which contain german umlauts. System encoding is set to de_DE.utf8 too.
Here is a small example:

prod=# \l
                                  List of databases
   Name    |  Owner   | Encoding |   Collate   |    Ctype    |   Access privileges   
-----------+----------+----------+-------------+-------------+-----------------------
 bucardo   | postgres | UTF8     | de_DE.utf8  | de_DE.utf8  | 
 prod      | postgres | UTF8     | de_DE.utf8  | de_DE.utf8  | 


prod=# \d "Lager_Behaelter"
             Table "public.Lager_Behaelter"
       Column        |         Type          | Modifiers 
---------------------+-----------------------+-----------
 Behälterbezeichnung | character varying(20) | not null
 Fachgröße           | character varying(40) | 
 VPE                 | smallint              | default 0
Indexes:
    "Lager_Behaelter_PrimaryKey" PRIMARY KEY, btree ("Behälterbezeichnung")
Triggers:
    bucardo_add_delta AFTER INSERT OR DELETE OR UPDATE ON "Lager_Behaelter" FOR EACH ROW EXECUTE PROCEDURE bucardo."bucardo_add_delta_Behälterbezeichnung"()
    bucardo_triggerkick_prod AFTER INSERT OR DELETE OR UPDATE OR TRUNCATE ON "Lager_Behaelter" FOR EACH STATEMENT EXECUTE PROCEDURE bucardo.bucardo_triggerkick_prod()



postgres@test:~$ bucardo_ctl list tables |grep public.Lager_Behaelter
Table: public.Lager_Behaelter                                DB: prodMA  PK: Behälterbezeichnung (varchar)

postgres@test:~$ bucardo_ctl list tables |grep public.public.Mängelberichte
Table: public.Mängelberichte                                DB: prodMA  PK: Mängelbericht (int4)

unequal number of entries in row expressions

ID: 104
Component: General
Version: unspecified
Date: 2012-06-07 16:35 EDT
Author: Chris ([email protected])


Bucardo 4.99.5 but the problem started on 4.99.3

I tired, losing my mind.

I blew out the table and started fresh, I did the same for bucardo and started
fresh. It used to work and now it's freaking out. I can't seem to get the
full query to see what the problem is. I turned line numbers to try and get
clues but no luck.

(6728) [Thu Jun  7 16:06:43 2012] #7275 CTL Created new kid 7537 for sync
"web_sync_eqweb"
(7537) [Thu Jun  7 16:06:43 2012] #1925 KID New kid, sync "web_sync_eqweb"
alive=1 Parent=6728 PID=7537 kicked=1
(7537) [Thu Jun  7 16:06:44 2012] #3087 KID Delta count for
web_o1.public.eqweb_event : 9
(7537) [Thu Jun  7 16:06:44 2012] #3793 KID Warning! Aborting due to exception
for public.eqweb_event:? Error was DBD::Pg::db pg_result failed: ERROR: 
unequal number of entries in row expressions\nLINE 1: ...c.eqweb_event WHERE
(eventid,changetime,username) IN ( ('201...\n                                  
                          ^ at /usr/local/share/perl5/Bucardo.pm line 8323.
(7537) [Thu Jun  7 16:06:44 2012] #2238 KID DBD::Pg::db pg_result failed:
ERROR:  unequal number of entries in row expressions\nLINE 1: ...c.eqweb_event
WHERE (eventid,changetime,username) IN ( ('201...\n                            
                                ^ at /usr/local/share/perl5/Bucardo.pm line
8323. Line: 3794 Main DB state: ? Error: none DB web_o1 state: ? Error: none DB
web_s1 state: 42601 Error: 7
(7537) [Thu Jun  7 16:06:44 2012] #7382 KID Kid 7537 exiting at cleanup_kid.
Sync "web_sync_eqweb" public.eqweb_event Reason: DBD::Pg::db pg_result failed:
ERROR:  unequal number of entries in row expressions\nLINE 1: ...c.eqweb_event
WHERE (eventid,changetime,username) IN ( ('201...\n                            
                                ^ at /usr/local/share/perl5/Bucardo.pm line
8323. Line: 3794 Main DB state: ? Error: none DB web_o1 state: ? Error: none DB
web_s1 state: 42601 Error: 7

# Table
CREATE TABLE eqweb_event (
    eventid     text NOT NULL,
    changetime  bigint NOT NULL,
    username    text NOT NULL,
    datetime_e  text,
    datetime_f  text,
    epochal     bigint,
    solid       text,
    lat         real,
    lon         real,
    depth       real,
    depthtype   text,
    depthunit   text,
    mag         real,
    magtype     text,
    felt_e      text,
    felt_f      text,
    location_e  text,
    location_f  text,
    report_url  text,
    PRIMARY KEY (eventid, changetime, username)
) 

# Setup

bucardo add database web_o1 host=### dbname=### dbuser=### dbpass=###
bucardo add database web_s1 host=### dbname=### dbuser=### dbpass=###
bucardo add dbgroup web_all web_s1:source web_o1:source

bucardo add herd web_eqweb eqweb_event
bucardo add sync web_sync_eqweb herd=web_eqweb dbs=web_all

bucardo validate all

bucardo stop
bucardo start

bucardo kick web_sync_eqweb

( after some time )

======================================================================
Last bad             : Jun 07, 2012 16:06:44 (time until fail: 1s)
Sync name            : web_sync_eqweb
Current state        : Bad
Source herd/database : web_eqweb / web_o1
Tables in sync       : 1
Status               : active
Check time           : none
Overdue time         : 00:00:00
Expired time         : 00:00:00
Stayalive/Kidsalive  : yes / yes
Rebuild index        : No
Ping                 : yes
Onetimecopy          : No
Post-copy analyze    : Yes
Last error:          : Failed : DBD::Pg::db pg_result failed: ERROR:  unequal
number of entries in row expressions
LINE 1: ...c.eqweb_event WHERE (eventid,changetime,username) IN ( ('201...
                                                             ^ at
/usr/local/share/perl5/Bucardo.pm line 8323. Line: 3794 Main DB state: ? Error:
none DB web_o1 state: ? Error: none DB web_s1 state: 42601 Error: 7  (KID 7537)
======================================================================

activate / deactivate doesn't obviously work

ID: 123
Component: General
Version: HEAD
Date: 2013-06-07 12:09 EDT
Author: Joshua Tolley ([email protected])


% bucardo deactivate sync cdc_prod_sync
Deactivating sync cdc_prod_sync
% bucardo status cdc_prod_sync
======================================================================
Last good                : Jun 07, 2013 10:06:31 (time to run: 9s)
Rows deleted/inserted    : 0 / 2
Sync name                : cdc_prod_sync
Current state            : Good
Source relgroup/database : cdc_prod_relgroup / cdc_prod_useast
Tables in sync           : 43
Status                   : Active
Check time               : None
Overdue time             : 00:00:00
Expired time             : 00:00:00
Stayalive/Kidsalive      : Yes / Yes
Rebuild index            : No
Autokick                 : Yes
Onetimecopy              : No
Post-copy analyze        : Yes
Last error:              :
======================================================================

You can tell from the process list that the controller for the sync has died,
but bucardo list sync and bucardo status both continue to claim it's active, as
does the sync table in the bucardo database.

onetime Does Not Work in 4.99.6

Tried to set up a two master configuration where one master already had data and the other didn't. Hit reload config and the daemons fired up, which is great, but nothing was copied. I changed a row in the populated master, and the updated data was copied to the other, but none of the existing data was. The onetime attribute was set to 0 after that.

Probably something simple…

Can't locate boolean.pm in @INC

Hello
New error message :)
Can't locate boolean.pm in @inc (@inc contains: /usr/local/lib/perl5/5.12.4/BSDPAN /usr/local/lib/perl5/site_perl/5.12.4/mach /usr/local/lib/perl5/site_perl/5.12.4 /usr/local/lib/perl5/5.12.4/mach /usr/local/lib/perl5/5.12.4 .) at /usr/local/lib/perl5/site_perl/5.12.4/Bucardo.pm line 33.
BEGIN failed--compilation aborted at /usr/local/lib/perl5/site_perl/5.12.4/Bucardo.pm line 33.
Compilation failed in require at /usr/local/bin/bucardo line 897.

Freebsd 8.2
p5-DBD-Pg-2.18.1_1 Provides access to PostgreSQL databases through the DBI
p5-DBI-1.616_1 The perl5 Database Interface. Required for DBD::* modules
p5-DBIx-Safe-1.2.5 Safer access to your database through a DBI database handle

v 4.99.3, ver 4.53 is ok

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.