radioml / dataset Goto Github PK

Can you please guide me with the steps how u are executing this file in gnu radio?
Thanks in advance

Traceback (most recent call last):
File "/home/winlab/PycharmProjects/GeneralGNU/dataset-master/generate_RML2016.10a.py", line 42, in
mod = mod_type()
File "/home/winlab/PycharmProjects/GeneralGNU/dataset-master/transmitters.py", line 35, in init
transmitter_mapper.init(self, mapper.BPSK,
AttributeError: 'module' object has no attribute 'BPSK'

There is a error. I've tried several versions of mapper but it still failed. What is the proper and correct version of mapper?

    Hi,Tim, I downloaded the dataset from the DeepSig Inc website, I found that most of the analog modulation data in the dataset is very strange, their IQ graphics are like this:

, I am curious, is this normal?

The classes.txt file in the dataset is in the following order:
classes = ['32PSK',
'16APSK',
'32QAM',
'FM',
'GMSK',
'32APSK',
'OQPSK',
'8ASK',
'BPSK',
'8PSK',
'AM-SSB-SC',
'4ASK',
'16PSK',
'64APSK',
'128QAM',
'128APSK',
'AM-DSB-SC',
'AM-SSB-WC',
'64QAM',
'QPSK',
'256QAM',
'AM-DSB-WC',
'OOK',
'16QAM']

I'm not sure if that is the same order as in the dataset.

Has someone figured this out?

I use the generate_RML2016.10a.py to generate the data set, which is the same number as the data set published by the author. And I use the same network to train the two data set, but the test accuracy are different. The accuracy of the data set generated by the code is 4-5 percentage points lower. Why? Are the parameters in the code provided by the author different from those in the public dataset?

The document in gnuradio.channels.dynamic_channel_model indicates that the -noise_amp – Specifies the standard deviation of the AWGN process .
So, I think there should be noise_amp = np.sqrt(10**(- snr/10)), original code is the variance.

Besides, the Normalization of the energy may be the energy=np.sqrt(np.sum(np.abs(sample_vector)**2)),sample_vector = sample_vector / energy, rather than the following form, the former formula can promise after be normalized the total energy of a complex vector equals to one.

I'm interested in determining the dataset's data rate or transmission rate. Can you assist with this?

Has anyone tested with real data?
I found that the prediction result is very inaccurate, does this only exist in the theoretical stage

After cloning the repo, when I try to run build_dataset command from the terminal (I'm using ubuntu 18.04), I get the following error:-

I think the radioml/minsdr image on the docker hub either doesn't exist or is still private.
I'm trying to run build the dataset using yonidavidson/minsdr docker image from @yonidavidson's docker hub.

I am trying to build the datasets but "import mediatools" fails.
I tried to install manually gr-mediatools but It doesn't work.

Hello, i am working on signal classification problem using RADIOML 2016 dataset.I need to calculate signal features like gamma max , kurtosis and many of the other expert features. Since the signal points are complex , i have a doubt if i should take absolute value . I am new to the area of signal processing and a undergraduate student . It would really helpful if u could help me with calculating the expert features using these complex signal points.
thank u

I'm trying to run the code with the Anaconda - Python (windows). It says "module gnuradio is missing". There is no "gnuradio" package available - Windows version at:

https://anaconda.org/ryanvolz/gnuradio

Should I use it with Linux? Any other solution would be appreciated.

When I excecuted 'python generate_RML2016.10a.py', errors occurred when generating am, fm and amssb mod:

[mp3 @ 0x7f8130000c60] Skipping 0 bytes of junk at 70700.
[mp3 @ 0x7f8130006380] Header missing
error decoding file

I think the my gnuradio 3.7 config is probably not right. How to deal with this?
Thanks!

Could someone tell me how to generate my own dataset with different channel parameters ??
I couldn't find any documentation on this.

When I run generate_RML2016.10a.py, this error come out:

$ ./generate_RML2016.10a.py
snr is -20
len(interp_taps) = 287
[mp3 @ 0x7fcfb8000b60] Skipping 0 bytes of junk at 70700.
[mp3 @ 0x7fcfb8002040] Header missing
error decoding file
...
error decoding file
WARN: fractional_interpolator is deprecated. Please use fractional_resampler instead.
[mp3 @ 0x7fcff0000b60] Skipping 0 bytes of junk at 70700.
[mp3 @ 0x7fcff0096c80] Header missing
error decoding file
all done. writing to disk

Is there something that my computer missed?

By the way, it seems that the RML2016.10a_dict.dat has also been generated successfully. The .dat file size is 640.9MB.

Hi there!

I've realized that in the dataset generation, the energy of the 128-sample data vectors is being normalized to unity as follows (lines 65 and 66):

However, to the best of my knowledge, the energy Es of a discrete-time signal x(n) is defined mathematically as:

Once you have calculated the Es, the sampled_vector must be divided by the square root of the energy, not only by the energy itself. In code, it should be something like this:

energy = np.sum(np.abs(sampled_vector) ** 2)
sampled_vector = sampled_vector / math.sqrt(energy)

I've plotted both versions and these are the results.

Before:

After:

Therefore the signals are being unnecessarily compressed, which can make it harder for (some) models to extract meaningful information, or even prevent it altogether.

Do my findings make sense to you or is there anything that I may have not understood properly? Please, check it and update us with your conclusions if you are so kind.

I look forward to hearing from you.

Regards,

Ramiro Utrilla

When I run generate_RML2016.10a.py, this error come out（only when generate am&amssb&fm）

$ ./generate_RML2016.10a.py
snr is -20
len(interp_taps) = 287
[mp3 @ 0x7fcfb8000b60] Skipping 0 bytes of junk at 70700.
[mp3 @ 0x7fcfb8002040] Header missing
error decoding file
...
error decoding file
[mp3 @ 0x7fcff0000b60] Skipping 0 bytes of junk at 70700.
[mp3 @ 0x7fcff0096c80] Header missing
error decoding file
all done. writing to disk

Is there something that my computer missed?
these three modulations signal data is wrong when I inspect frequency spectrum and power spectrum。

I got this error when I run generate_RML2016.10a.py, who can help me, many thanks

Traceback (most recent call last):
File "generate_RML2016.10a.py", line 42, in
src = source_alphabet(alphabet_type, tx_len, True)
File "/home/niejinbo/PycharmProjects/dataset-master/source_alphabet.py", line 35, in init
self.src = mediatools.audiosource_s(["source_material/serial-s01-e01.mp3"])
AttributeError: 'module' object has no attribute 'audiosource_s'

I dont know how to solve this, where is this modules?
If anyone knows the solution, please let me know. Thank you very much!!!!

What is the carrier frequency of the modulation modes in the RML201610A dataset?

Has anyone designed a blind equalization algorithm on this dataset, I have tried algorithms such as CMA/MMA, but the results are not ideal. Can anyone share the experience?

Hello When Try to import the data set which is in .pkl format in python 3.7 it shows error

Hi developer,
I was trying to run generate_RML2016.10a.py, and the following run time error occurs:
Mapper warning: Could not load the module “fastcluster”.
The module “scipy.cluster.hierarchy“ is used instead, but it will be slower.
The 'cmappertools' module could not be imported.
The 'cmappertools' module could not be imported.
Intrinsic metric is not available.
The 'cmappertools' module could not be imported.
Traceback (most recent call last):
File "generate_RML2016.10a.py", line 3, in
from source_alphabet import source_alphabet
File "/root/dataset/source_alphabet.py", line 3, in
import mediatools
File "/usr/local/lib/python2.7/dist-packages/mediatools/init.py", line 45, in
from mediatools_swig import *
ImportError: No module named mediatools_swig

I have already installed gr-mediatools from source (saw a similar issue there but no response exists).
Please help.

Hey guys,

I'm trying to run the dataset generation tools from outside of the docker environment that you have setup I am getting an error in the transmitters.py:

ImportError: No module named mapper

I presume there is a dependency that I'm missing here. I noticed in the docker environment / build_dataset script that your running a setup_env.sh which I presume is adding some paths to the python path for execution.

Thanks for any help

When I run the program generate_RML2016.10a.py, the data can't be generated, so I checked and found snk.data() is None, it was cased by follows:

snk = blocks.vector_sink_c()
raw_output_vector = np.array(snk.data(), dtype=np.complex64)

I print the snk.data and I got a '()' and the length of raw_output_vector is 0, who can help me to fix this problem, many thanks

When running the data generation script, I am missing files

source_material/gutenberg_shakespeare.txt: No such file or directory
Traceback (most recent call last):
  File "/home/aaron/src/radioml/dataset/generate_RML2016.10a.py", line 39, in <module>
    src = source_alphabet(alphabet_type, tx_len, True)
  File "/home/aaron/src/radioml/dataset/source_alphabet.py", line 13, in __init__
    self.src = blocks.file_source(gr.sizeof_char, "source_material/gutenberg_shakespeare.txt")
  File "/home/aaron/src/gr/lib/python2.7/dist-packages/gnuradio/blocks/blocks_swig0.py", line 436, in make
    return _blocks_swig0.file_source_make(*args, **kwargs)
RuntimeError: can't open file

I do not appear to have access to the dependent submodule.

$ git submodule update
Cloning into 'source_material'...
ERROR: Repository not found.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
fatal: clone of '[email protected]:radioML/source_material.git' into submodule path 'source_material' failed

Are these files necessary or there is a way to bypass them (use random number generator maybe?)

I think the noise amplitude calculation should be:
10^(-snr_db/20.0), but the code has: 10^(-snr_db/10.0)

It looks like all the transmitters scale to an average magnitude squared P_avg=1. Then SNR_lin = P_avg/sigma^2, so sigma^2 = 1/snr_lin = 1/(10^(-snr_db/10)) => sigma = sqrt(1/(10^(-snr_db/10))) = 10^(-snr_db/20). The dynamic channel model takes in the standard deviation not the variance.

What am I missing?

(dp1
(S'QPSK'
p2
I2
tp3
cnumpy.core.multiarray
reconstruct
p4
(cnumpy
ndarray
p5
(I0
tS'b'
tRp6
(I1
(I1000
I2
I128
tcnumpy
dtype
p7
(S'f4'
I0
I1
tRp8
(I3
S'<'
NNNI-1
I-1
I0
tbI00
S'!a\xc1\xbbK\xbc\x19\xbb7PC\xba0+\xaf\xbb(\xb5\xbd\xbb\x99Fr\xbbB$\xa3\xbbR%\xd7\xbb\x9cC\x14\xbc\x11}\x9e\xbb)6%\xbc\xd2\xb4\x94\xbb\xbfd\x8d\xbb\x11=\xa8\xbbp\x16\xc7\xbb\xaa\x89\x9b:=\x97\x98\xbb\xbd\xd8\xe3\xbb4\x81\xda\xbb\xba\x04\xd3\xbbD\xb2z\xbb\xbdn\t\xbc\xcd\n&\xbc\x89\x1f\xe2\xbbh\xc0\x1d\xbcY\xa0\xcb\xba$\r=\xbb\xcb\n\x94\xbb\xf2~;\xf8\xd3\xf2;\xbf\x0c\;\xdd\xe5\xf9;B\x9e\x10<m\xac;WH\xf1:\x1c?\xd2;\r=+\xb9\xc2\xe3\x12\xbb\xdb#\xaa\xbb\xcf\x7fn\xbb&\xff%\xbc\x04\xb6\xd1\xbbl\xb9\xc6\xbb\x84/\xfb\xbb\xcf\xfda\xbbs\x03\xe89o>(;\x9f}\x9b;$\xcf\x98;\x86@\xd4;a\xe9\x0b<\x9cU\xa3;\x83w@9T\xbd\x849\xc1\x19>\xbb-\x9b\xaf\xbb4\x81\x18\xbc%\x93\xa5\xbb\xda\x97\x9e\xbb\xc9h\xf78j\xe2\x0f\xba\xe8\x9df:h\x18\xdf;\xcao\x03<\xf4\xfc\x05<a@\xe0;io[;\x99\xb7+;^y$\xbb\xcb\xb8\x1f\xbaJ"\x03\xbcL\x1a/\xbc\x80"\x06\xbc\x0f\xfb\x1b\xbc\x8e\\\x98\xbb\xba\x89\x97\xba\x0c\xa2\x10;.]\xa3;\'\x0f\x0e;\xf7H\xd0;SY+<\x05E\xf5;\xb0X\xcc;\xd8\x7f@;p\x9c\x98:\x15\xf0r9MR\xa0\xbb\xed\xd2Y\xbbo\x8d\xd8\xbb_\xfe\xa0\xbb,\xf4\xee\xbb\x1e\x1c\xc5\xbb\x17\xc4\xa8\xba\xa9Z\xdd9}\x0e\x8a93\x93I;\xd1\xa8\x82;\xeb\xc1;^\xa20;\x06\xe9\xe9;t\x9e\xcc;\xf1\x97\xc8;I\xba\xb1;:\xfe\x83:5\xcd\x80:\xcc\x8c\xb3:H\xbcg;\x99\x10\x8d;T\x99\t<\xf8\x14\x03<\xef\x80\xda;\xcf%\x07<\x06\xd3;Z[0<>\xf1\xde;\xae?2;a\xa4\x8c8\x872\x10;f\xbez:{\x87\xba\x87U\xae\xbb\xc5\xde\xf2\xbb\xdc\xee\xee\xbbO\xdd\xe58\xfa__\xbb\xe4/V\xbb\xf4RG\xbb\xccu\xba;\xc2q\xff\xbb@\x10\x00\xbci\xb7\x83\xbb7\x8f\xa7\xbbT\xa0\xc2\xbbQH\xd7\xbb\x9f\x00\xb7\xbb@\x98\xad\xbb\xcfs\9\x00(\xc6\xbb\xcd\x1c\xe7\xbb\x15\xc8\xfb\xbbs\xc9\xdf\xbb/w\xac\xbb?d\x8c\xbbT\xed\x06\xbcD{\x11\xbc\xaa\x1c\xda\xbb^\x13\x0f\xbc\xf1\x16\x88\xbbY\xaa\x05\xbc\x15\x7f\xc5\xbb<\xb5T\xbc\xfc\x0b\xe1\xbb\xaf(\xdd\xbb\xa0J\x84\xbb\xa6\x11\xff\xbb\xc4\x1f\xae\xbbL~R;A\xff\xa3\xbb-D\x9f:\xcf\xe4=:\x89R\x91;'1\x08<m\xb4\x10<\xad*\xc2;\xfd\x9c\x08<\n\x89\xeb;\x87\xb6\x05<c\x9d\x0e<\r!\xd5;\x98\xfe\x8f;;!9;S\xcc\r;d\x80\xaa;X\xae\xf2;W\x04\x0b:b\xb4\xa4;\xb6*\x9d;\x92'\x12<\x9d\x89\xee;\xd8\xa3\x86;\xea\x13\xd7\xba\xb7\xa4\xa89\x11\x8e\x8e\xbbs\xfd\xae\xbb\xf1A\xdc\xbb2\x8f\x86\xbb\xa1\x1b\xae\xbb\xf6\x9f\x95\xbb\xf9\x03\x9c\xbb\xec\xc4+\xbbd-;\xf5\xb4\x01<\\xbc\x04<\x16C\x0c<\x82O\xc0;u\xd0\x14<!B6;\xd7"\x96\xbb/\xacM\xba\xcfJ\xf9:R\xae\xa8\xbb\xde\x85\x1e\xbc\xd6\x16\xf2\xbb\x8bY\x0f\xbcU\xc6\x90\xbb\xcc\xda\xe1\xba\xed\xa8J\xbb\xc5S]\xba\xfd:\xe6;WA\x1b<GJp;){\x16<\x8d\xe1\x8e;\x05\xbd\x06<GO\t<\x95(\xed;\x92\xccH;U\xef\xf4;>\x8b?;7\x90\xa5;$\xcc\xad9L\x85\x18<\xef\xaf\x97;\xc5\xcf\xd7;\x99\xc4\xb3;w\x82\xd1;"\x12\xa6;?b\xb6;'w\xdf;w+\x15<O\x81\t<\xca\xdd\x0e<\x9c,&<\xd5\x8c\xf8;q\xc8\x9d;\x16R\xb9;\xc3\x1c.;\x1bi\x1a\xbb\xfb1\xa4\xbbF\x177\xbbA\x10\x9e\xbbQ\x06\x92\xbb\xca\x17\x0c\xbbA\x06\xe0\xba\xe5\x834\xbb\x8f\xdb\x0b\xbb\x18Y\xae\xb9!\xe0\x11\xba\xd2\x87\x04<8V\xda;\xf5\xbf\xef;9\x9f\xb5;\xfc>\xaf;r\x1d)<O\xdf\t<\x03p\xb2;\xa3\xe8\xa4;Viw:q&\xf8\xba\x7f\xf2\xac\xbb\x0bi\x17\xbaQ\xb2\xba\xbb\xaf\xa1\xcd\xbb\xd8\x9a\x1f\xbc\x12"\x1f\xbc\xc4\x15\x11\xbc\xaaD\xec\xbb\xac\xfd\xe3\xbb!\xef\x01\xbcb\xc9\xd7\xb8\t\x14+\xbb\xb3{_\xbb|\xb8D;\x10\x85\xed\xb9\xafw\xbe:\xd3\t\xf5;\x9a\xadk;j\x9b\xf3;\xd2\xe1\xbd;\xddv\x0c<\xdd\x19\xb9;M\xef\xa2;\xd0\x96\xe9;\xc3e3;']\xe0;\x158\xfd:\x8b\xe3#;ii\x93;\xc8\xb9\xdc;%\xf6\xc7;\xd0Y\x92;<rx;"\xe05;\xb6\r\x98;K\xb4\xb0;\xee\xb5;\xb7\xa5\xe1;K\xf7\xa1;\xf31\xf0;`\x07\x02<sP\x93;pe;;

the data is what?

Sorry, when I tried to run the generate_RML2016.10a.py, this error came out. Could you please give me a hand?

Traceback (most recent call last):
File "e:\python\dataset_master\generate_RML2016.10a.py", line 3, in
from source_alphabet import source_alphabet
File "e:\python\dataset_master\source_alphabet.py", line 3, in
import mediatools
ImportError: No module named mediatools

I want to how to plot the constellation diagram of the energy units signals？

Hi Tim,

I was wondering if you've published (or plan to publish) the dataset + codebase associated with "End-to-End Radio Traffic Sequence Recognition with Deep Recurrent Neural Networks." I'd very much like to tinker with that kind of data (mostly look more closely at injecting additional frontend non-idealities and comparing classification performance with a baseline), and it'd be super helpful if I could use your work as a starting point.

Thanks!

Angie