sleinen / samplicator Goto Github PK
View Code? Open in Web Editor NEWSend copies of (UDP) datagrams to multiple receivers, with optional sampling and spoofing
License: GNU General Public License v2.0
Send copies of (UDP) datagrams to multiple receivers, with optional sampling and spoofing
License: GNU General Public License v2.0
Is it possible to EXCLUDE source hosts/networks such that their messages are NOT forwarded?
Example:
I have some hosts (firewalls) that send GBs of log messages every day and I don't want to forward those messages to our Solarwinds server, because it simply can't handle the load.
$data _center_network: 10.1.0.0/255.255.0.0
$firewall: 10.1.1.10
$solarwinds: 10.1.2.3
So, I'd like to be able to keep my configuration very simple by having something like this in my config:
$data _center_network:$solarwinds/514, !$firewall
I realize that I could just define every host I want logs forwarded from in the samplicator config, but that means I have to edit that config every time I have a new or changed router or switch.
From [email protected] on October 19, 2011 21:54:36
Feature request: it would be nice if samplicate had a way to support multiple ports with a single instance of samplicate. Note that I don't need this today, but foresee needing it in the future. [So please set priority on this to low!]
Original issue: http://code.google.com/p/samplicator/issues/detail?id=15
Hello,
I'm trying to debug Netflow V9 from CISCO without source_id value
The goal is to add IP source (router who generates flow) to the source_id https://netflow.caligare.com/netflow_v9.htm before duplication.
This can be very useful to find the source device
Any advice on how to do this would be greatly appreciated
For now I'm trying to see the buffer without any success
https://github.com/sleinen/samplicator/blob/master/samplicate.c#L346
I added this right after
if (n > 0) { fpdu[n] = '\0'; printf("Test received message: %s size: %ld\n", fpdu, sizeof (fpdu) ); }
But fpdu message is always empty (size not of course), I don't understand why
I guess there is something specific here, a basic client/server UDP test works well
I guess that show the socket data can also be very interesting for debugging purpose in many cases
Thanks
Hello there, I would like to know if it is possible redirect packets keeping the original IP and an alternate source IP at the same time with this tool.
I'm currently running samplicator 1.3.6.
i.e:
source Net: 10.0.0.X / 25
Samplicator Server: 10.2.0.1
Destination Server 1 - IP: 172.27.0.1
Destination Server 2 - IP 10.1.0.1
FWD packets to server 1 with an alternate Source IP. (preferable the IP of the server where samplicator tool is running)
FWD packets to server 2 with the original Source IP.
Thanks!
From [email protected] on September 18, 2011 19:43:11
What steps will reproduce the problem? 1. Call samplicate with the "-x " and "-c " options
What is the expected output?
The outgoing packets to multiple destinations should be spaced by microseconds.
What do you see instead?
They are sent back-to-back.
Original issue: http://code.google.com/p/samplicator/issues/detail?id=8
drawks@oingo$ file ._acconfig.h
._acconfig.h: AppleDouble encoded Macintosh file
Noticed while building some debian packages that these little binary crumbs are all over dist tarball.
From [email protected] on April 22, 2012 16:49:21
John Heasley points out that arguments to setsockopt() should be int, but the current code passes longs. This would make the code fail on 64-bit machines.
I must say that I have never noticed this, and I have been running samplicator on 64-bit GNU/Linux machines for quite some time. But it is plausible that this would cause problems on other systems.
Original issue: http://code.google.com/p/samplicator/issues/detail?id=24
From [email protected] on March 14, 2012 18:44:23
-S doesnt work on intel/amd solaris. I have attached a patch for review.
Attachment: rawsend.c.patch
Original issue: http://code.google.com/p/samplicator/issues/detail?id=20
From [email protected] on April 06, 2011 16:51:22
Hi,
We have many flow packets that arrive from a few hundred routers and the need is to duplicate some of the packets to different destinations without changing the dst ip on the routers.
so we used
samplicate -p 9996 -f -S 1.1.1.1/9996 2.2.2.2/9996
but then the need came up to duplicate certain routers to additional ones or remove some from the 2 already defined.
so we started using:
samplicate -p 9996 -f -S -c /etc/samplicate.conf
samplicate.conf:
5.5.5.5:1.1.1.1/9996 2.2.2.2/9996 3.3.3.3/9996
0.0.0.0/0.0.0.0:1.1.1.1/9996 2.2.2.2/9996
Now while doing a top I saw that the cpu spikes are getting quite high from samplicate and I wonder if this is really a problem or can this be ignored.
We want to add more destination and maybe remove certain destinations from matching the 0.0.0.0 rule.
I wonder if it would be easier to specify some kind of catch all without the program having to see if the 0.0.0.0 line matches?
Original issue: http://code.google.com/p/samplicator/issues/detail?id=5
From [email protected] on March 22, 2012 16:02:58
Hi,
Is it possible to do load balancing using the frequency parameter ?
Case:
10K requests come to Samplicator, it needs to divide to Server A 5K, Server B 5K requests. (Servers have application to send back responds.)
BR,
Ali KIZIL
Original issue: http://code.google.com/p/samplicator/issues/detail?id=22
From [email protected] on October 19, 2011 21:59:17
According to a comment on issue #1
, IPv6 support works at a basic level but does not support spoofing. This is an enhancement request for spoofing support. Note that I don't need IPv6 support at all today, let alone IPv6+spoofing, but anticipate needing it in the future.
Original issue: http://code.google.com/p/samplicator/issues/detail?id=17
PR #48 introduced a construct that isn't understood by all C compilers:
for (int j=0; j<argc; j++)
I should have reviewed this better!
It says that the port separator is ':', but it's '/'.:
https://github.com/sleinen/samplicator/blob/37e0e90cec9021ecf287e5de0c0dc09b181b27e4/read_
config.c#L415
From [email protected] on October 19, 2011 21:56:17
Many daemons support reloading the config file on SIGHUP. samplicator should, too.
Original issue: http://code.google.com/p/samplicator/issues/detail?id=16
On FreeBSD 11-RELEASE-p1 with samplicator 1.3.8.rc1
Running samplicator with the following syntax no longer works. No flows are captured
$ samplicator -p <port> <target>/<port>
This however, does work
$ samplicator -s 0 -p <port> <target>/<port>
Samplicator is also no longer able to spoof source address. This seems to be due to a change in FreeBSD 11 to no longer convert incoming packet headers ip_len and ip_off fields to host order. They now remain in network order with the rest of the packet header
Hi,-
in a large environment with multiple destinations, it would make sense simplicator to put the flow on multicast streams. i would reduce the processing power and bandwidth.
Any thoughts ?
thanks,
idsteiner
From [email protected] on October 19, 2011 21:24:14
What steps will reproduce the problem? aclocal
autoheader
automake -a
autconf
./configure
make What is the expected output? What do you see instead? autoheader produces warnings:
autoheader: WARNING: Using auxiliary files such as acconfig.h',
config.h.bot'
autoheader: WARNING: and config.h.top', to define templates for
config.h.in'
autoheader: WARNING: is deprecated and discouraged.
autoheader:
autoheader: WARNING: Using the third argument of AC_DEFINE' and autoheader: WARNING:
AC_DEFINE_UNQUOTED' allows one to define a template without
autoheader: WARNING: acconfig.h': autoheader: autoheader: WARNING: AC_DEFINE([NEED_FUNC_MAIN], 1, autoheader: [Define if a function
main' is needed.])
autoheader:
autoheader: WARNING: More sophisticated templates can also be produced, see the
autoheader: WARNING: documentation.
automake -a produces warnings:
automake: configure.in: installing ./install-sh' automake: configure.in: installing
./mkinstalldirs'
automake: configure.in: installing ./missing' automake: configure.in: installing
./config.guess'
automake: configure.in: installing ./config.sub' Makefile.am:7: @LIBOBJS@ seen but never set in
configure.in'
make produces:
gcc -DHAVE_CONFIG_H -I. -I. -I. -g -O2 -c samplicate.c
samplicate.c:38:18: error: inet.h: No such file or directory
make: *** [samplicate.o] Error 1
I have arpa/inet.h, so I tried making the #include "inet.h" to be in a #else to the arpa/inet.h check. This was in samplicate.c and read_config.c. I got farther, but then make yielded:
make: *** No rule to make target inet.o', needed by
samplicate'. Stop. What version of the product are you using? On what operating system? r126 , from the repository, on Linux -- Debian 6.0.2. Please provide any additional information below. autoheader version 2.67.
automake version 1.4-p6.
Original issue: http://code.google.com/p/samplicator/issues/detail?id=12
I don't see any LICENSE file in the repo, nor any obvious mention to license in the README
Skipping this issue number to maintain synchronization with Google Code issue IDs.
Original issue: http://code.google.com/p/samplicator/issues/detail?id=9
From [email protected] on August 06, 2010 11:47:40
In our company we need to have always same source port for samplicated packets, so I wrote a small patch, which allows that.
I think it might be useful also for other users, at least as a optional patch.
Patch adds option to set source port into receiver settings.
A.B.C.D[/port[/freq][,ttl][.src_port]]
It's not optimal, because Samplicator uses only one socket for all outgoing packets so source port is set only once, when first modified receiver is found.
Source port settings are ignored in spoof mode (-S).
Attachment: samplicator-1.3.6-src_port.patch
Original issue: http://code.google.com/p/samplicator/issues/detail?id=3
From [email protected] on March 14, 2012 22:30:17
Purpose of code changes on this branch: Fix for issue #20
When reviewing my code changes, please focus on: Whether it could break any systems, or whether you can think of other systems than Solaris where this variant is necessary. After the review, I'll merge this branch into: /branches/1-3-maint
This is also in 1.3.7-beta3.
Original issue: http://code.google.com/p/samplicator/issues/detail?id=21
From [email protected] on March 13, 2012 14:54:57
Purpose of code changes on this branch: To support replication of large UDP datagrams. When reviewing my code changes, please focus on: Possible issues when sending out copies of large packets, especially when raw mode (i.e. spoofing) is used. After the review, I'll merge this branch into: /trunk Well, actually I guess I'll release this as 1.3.7. But maybe I get some other small changes in as well.
Original issue: http://code.google.com/p/samplicator/issues/detail?id=19
From [email protected] on March 11, 2009 09:48:02
What steps will reproduce the problem? 1. Start the samplicator on some port
2. Send UDP packets to that port over IPv6 What is the expected output? What do you see instead? Expected: The packets should be replicated to the configured destinations.
Instead, they are dropped by the host's network stack, because the
samplicator process uses an IPv4-only socket.
Similarly, when IPv6 addresses are specified as destinations, the
samplicator complains about malformed addresses.
Ideally, the samplicator should even be able to receive datagrams over IPv4
and distribute them over IPv6 and vice versa.
Original issue: http://code.google.com/p/samplicator/issues/detail?id=1
From [email protected] on September 19, 2012 05:22:21
What steps will reproduce the problem?
What is the expected output?
What do you see instead?
Expected is to see in vlc the image that ffmpeg is sending, but delayed for 10 seconds. What I see instead is a lot of dropped packets, without anything useful received at the vlc's side. If I lover the -x param for samplicator, to some really low value, like -x 1000 or such, then it works as expected. What version of the product are you using? On what operating system? 1.3.6 stable on debian squeeze (stable)/linux Please provide any additional information below. I'll provide if needed, but I just need to delay the udp stream for 10 seconds, that's all.
Original issue: http://code.google.com/p/samplicator/issues/detail?id=26
From [email protected] on October 19, 2011 21:50:35
This is an enhancement request.
Many programs come with an option to check the syntax of the config file. This is useful in init scripts. A common option name is -C. Request: equivalent option for samplicate.
Original issue: http://code.google.com/p/samplicator/issues/detail?id=14
In flow testing, it's sometimes beneficial to choose an alternate source IP.
Open ended:
Allow setting the IP via the command line.
Tighter controlled: (if you're worried about UDP floods)
Allow setting the IP from any given IP on the box. (primary or secondary ips)
I can not see configure file
i've seen two behaviors. When sending flow to mulitple collects i've experienced where samplicate will process the flow from one rule then stop. I've also seen it where it will process two rules.
Use case:
I have a rule for 192.168.10.0/255.255.255.0: 10.1.1.1/2055
I have a rule 0.0.0.0/0.0.0.0: 10.1.1.2/2055
some times when samplicate is running and a flow comes in for a device in 192.168.10.0/24 subnet it will just be sent to 10.1.1.1. then i'll restart the samplicate process and it will go to both destinations.
Can I get clarity on how the list is processed?
Hello,
I'm having a very hard time compiling Samplicator for Windows. I'm using Cygwin. Is there any Documentation or some Help file anywhere explaining how to Compile for Windows? If there is this would be a much appreciated.
The following
/usr/local/bin/samplicate -s 239.168.100.7 -p 5000 127.0.0.1/50405 127.0.0.1/50406
Doesn't work for me. 239.168.100.7:5000 is a multicast stream that I can read with VLC, etc. However samplicate doesn't seem to join the multicast group.
Hi,
Scenario - devices are sending syslog and netflow to one samplicator, so one udp port (514) for syslog and one udp port (2500) for netflow. Syslog & netflow saved into same device and also forwarded to two other devices.
Now I have need to forward traffic from only some sender syslog and/or netflow to third device, so samplicator internal filtering would be best for such purpose. Using regex for strings will definitely be (slight)performance issue, but using sender src IP for the filter, should be fine.
Can You please add such functionality?
Hello,
By default, samplicate intercept all on ipv4 interface ( 0.0.0.0) but not for ipv6.
So with ipv6 and multiple address, we must start multiple samplicate instance with -s
Regards,
Nicolas
From [email protected] on October 15, 2011 22:41:29
What steps will reproduce the problem? 1. Samplicator is running on a relay server with the following options.
./samplicate -p 162 -S -f -n x.x.x.x/162 where x.x.x.x is the final trap receiver's IP address.
Attachment: baltlmcss02_ivms2.pcap
Original issue: http://code.google.com/p/samplicator/issues/detail?id=11
From [email protected] on January 08, 2013 09:06:12
What steps will reproduce the problem? Command which is running on the server to forward all snmp traps to two other servers :
/usr/local/bin/samplicate -fS -p 162 192.168.0.197/162 192.168.3.181/162
The first server (197) seems to get all traps fine, but the second looses lots of traps. Those are 2 different monitoring machines working with the same configuration, code, and store the traps in the database. But the 2nd server (3.181) does not get all the traps.
This is now ongoing for about 4 weeks and because it is an internal network I can't think of loosing those packets on the cable internally.
But it is of course really lots of traps that get duplicated, around 30 - 40k per day.
I can also always see a time difference when the traps are entered in the database.
The first server always stores the traps around 20 minutes before the second server receives the trap and stores it. What is the expected output? What do you see instead? I am expecting that both machines would receive the traps around the same time and also receiving the same amount of traps. What version of the product are you using? On what operating system? OpenSuse 12.2 x64
If you need further details I will provide.
Original issue: http://code.google.com/p/samplicator/issues/detail?id=27
From [email protected] on April 16, 2012 12:16:02
I'm missing a version output of samplicate to see which samplicate version is currently installed on the system..
Original issue: http://code.google.com/p/samplicator/issues/detail?id=23
From [email protected] on October 19, 2011 21:47:37
The samplicator distribution does not seem to include an init script by default. I've written one, which you are welcome to distribute if you wish. Attached.
Attachment: samplicate
Original issue: http://code.google.com/p/samplicator/issues/detail?id=13
I am working with traffic from application which sends UDP datagrams longer than 1500 Bytes so they are fragmented in IP layer.
in samplicate.c PDU_SIZE is fixed to 1500 here
#define PDU_SIZE 1500
result of which - maximum 1500 bytes from buffer is copied and no warning nor error is thrown in case of longer UDP datagram.
I see that further there is some test that should catch longer datagram:
if ((n = recvfrom (ctx->fsockfd, (char*)fpdu, sizeof (fpdu), 0, (struct sockaddr *) &remote_address, &addrlen)) == -1) {
fprintf (stderr, "recvfrom(): %s\n", strerror(errno));
exit (1);
}
if (n > PDU_SIZE) {
fprintf (stderr, "Warning: %d excess bytes discarded\n", n-PDU_SIZE);
n = PDU_SIZE;
}
But at least in my case (kernel 2.6.18) it fails to catch it.
I think the reason is lack of MSG_TRUNC
flag in recvfrom()
call for modern kernels (since 2.2).
My workaround was changing PDU_SIZE to larger value, and it solved problem for my long UDP case but not sure if it won't broke something else - for example TCP handling.
From [email protected] on March 09, 2011 16:32:24
What steps will reproduce the problem? n/a What is the expected output? What do you see instead? the device should be displayed in the Netflow Tracker What version of the product are you using? On what operating system? Older version of Samplicator. Red Hat rel.5 is the platform Please provide any additional information below. Our Samplicator is running fine, however one thing I noticed is that when I am adding new devices it doesn't seem to display the new device to the Netflow Tracker. But for older device entries, the samplicator continuous to export data to the Tracker no problem at all. What could be a possible cause that hampers the samplicator to export new list of devices after adding to samplicator.conf file?
Original issue: http://code.google.com/p/samplicator/issues/detail?id=4
some hints for using samplicator to delay ffmpeg stream correctly ?
my build always destroys h264 data and creates LAG / fragments and other decoding errors.
cant use it in production, tried a lot of different bufsizes but i dont get it to work smoothly
From [email protected] on February 23, 2010 16:56:06
What steps will reproduce the problem? 1. Start samplicator as a daemon using `-f'
2. Look for a PID file
3. Look for an option to create a PID file
What is the expected output?
A PID file or an option to write one.
What do you see instead?
Nothing.
Original issue: http://code.google.com/p/samplicator/issues/detail?id=2
From [email protected] on March 28, 2014 18:57:41
What steps will reproduce the problem? I don't know, I followed the steps for installing the Samplicator application as documented. What is the expected output? What do you see instead? What version of the product are you using? samplicator-1.3.5
On what operating system? CentOS 6.5 Please provide any additional information below.
Attachment: Samplicator Issue with Forwarding to Receivers on port 2055.txt
Original issue: http://code.google.com/p/samplicator/issues/detail?id=28
Hi there! First, thank you for sharing this utility, it's really great!
I have noticed on a couple different VMs that I've tested this on (all building from latest source) that CPU utilization for a single core from the samplicate process is consuming ~90% CPU at all times, even with no data being sent to the service. Is this expected behavior? I haven't tried sending data to load it up yet.
Here are the flags I'm using:
/usr/local/bin/samplicate -S -f -p 2055 -c /etc/samplicator/samplicator.conf
And the config file:
# source/mask:destination/port
0.0.0.0/0.0.0.0:127.0.0.1/2055 10.5.2.39/2055 10.250.38.17/2055
From [email protected] on August 30, 2012 01:33:00
What steps will reproduce the problem? 1. Running samplicator in spoofing mode while trying to use the config file instead of the CLI. What is the expected output? What do you see instead? The expected behavior is to have a config file option to allow spoofing which presently doesn't seem to exist. The current behavior is insufficient because this mode can only be enabled via the CLI which causes a discrepancy between CLI versus config file modes of use. What version of the product are you using? On what operating system? 1.3.6 and 1.3.7 beta
OS: Redhat Linux Please provide any additional information below. We have provided a tentative patch to address this issue.
Original issue: http://code.google.com/p/samplicator/issues/detail?id=25
From [email protected] on August 08, 2011 10:58:32
I'm trying to use samplicate to get around the following issue:
We have one host that receives UDP data on one port, where lots of clients reports back to a specified address and specified port.
We don't have control over the applications that sends the data or the one that receives it, but we host the server that host the receiving application, and have root access to this.
Another application has been developed that also needs to listen to the same data. This could run on the same machine, or, preferrably on a different host.
I've tried running samplicate on the receiving machine, using the arguments; samplicate -p 1707 -S 192.168.x.x/1707 127.0.0.1/1707, where the 192.168.x.x-adddress is the host for the new app.
It forwards the packets nicely to the 192.168-host, but the local application does not receive any packets when using Samplicator in this manner.
Is it possible to forward packets on the same host and port that it receives UDP data on?
Original issue: http://code.google.com/p/samplicator/issues/detail?id=6
Skipping this issue number to maintain synchronization with Google Code issue IDs.
Original issue: http://code.google.com/p/samplicator/issues/detail?id=10
From [email protected] on September 02, 2011 21:02:51
What steps will reproduce the problem? 1. running Samplicate
2. on RHEL 4.1
3. in VMware What is the expected output? What do you see instead? the samplicate process crashes constantly. current work around is a cron job to check process and start if no running every 1 min. What version of the product are you using? On what operating system? 5 year old code Not sure of version. RHEL4 Please provide any additional information below. has anyone seen this behavioer before? and is this a known issue and does latest code fix this?
Original issue: http://code.google.com/p/samplicator/issues/detail?id=7
From [email protected] on March 11, 2012 08:33:49
What steps will reproduce the problem? Use samplicator to forward UDP packets with a size greater than 1500 bytes What is the expected output? What do you see instead? Expect packets bigger than 1500 bytes to be forwarded in full. Instead, packet is truncated to a maximum size of 1500 bytes What version of the product are you using? On what operating system? 1.3.6, Linux Please provide any additional information below. Could the #define PDU_SIZE in samplicate.c be increased to 65535 - the maximum possible UDP packet size.
Original issue: http://code.google.com/p/samplicator/issues/detail?id=18
Please add the ability to define the receiver by hostname and not just IP address. This would allow to forward to a cluster hostname using RR DNS or a service discovery hostname. This would increase the availability of the replicated packet.
Using just a VIP would require something like a hardware load balancer or managing an IP address with something like corosync. Being able to use modern techniques like service discover allows for an active-active load balance (although a harware LB ie. F5 would do this, I don't want to pay for one of those), which adds a scale out feature.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.