bavc / signalserver Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
The rename option raises a FieldError at /policy/rename with Cannot resolve keyword 'policy_name' into field. Choices are: filename, id, process, process_id, row, status, task_id
Repo needs an open source license declaration!
On the "Add/Delete Filter on Policies" as rules are edited some value doesn't show. For instance if saying that psnr.Y exceeds a cutoff number of 1. The '1' doesn't appear after the rule is supplied. There should be a way to view and edit existing rules of a policy (so far they can only be deleted).
Also Filter2 should only be shown contextually if average-difference is selected, else it has no meaning.
could be more elegant
FILES_VOLUME_PATH=/files ./quickstart.sh
./quickstart.sh: line 2: docker-machine: command not found
./quickstart.sh: line 3: docker-machine: command not found
./quickstart.sh: line 4: docker-machine: command not found
./quickstart.sh: line 5: docker-machine: command not found
./quickstart.sh: line 6: docker-compose: command not found
./quickstart.sh: line 7: docker-compose: command not found
./quickstart.sh: line 11: docker-compose: command not found
./quickstart.sh: line 11: docker-compose: command not found
./quickstart.sh: line 11: docker-compose: command not found
./quickstart.sh: line 11: docker-compose: command not found
I suggest storing the upload timestamp of new files supplied to the database and showing that timestamp on the file report.
Show file's record, along with what summarizations have been made and the resulting output of each summarization
creating a group is a multi-step process: upload individual files, search for individual files that you just uploaded, create group
suggesting a more simplified approach:
choose to upload either a directory or individual files (as in MediaConch)
at the prompt, choose if you want to create a new group or add to an existing group
possibly the name of the directory can default as the new group name
provide a ui so that the user may create, edit, delete, import, export evaluation criteria by selecting from a list of unique metadata keys (BRNG, TOUT, etc) and an operator such as average, count of frames greater than X (or equal to, less than, etc),
When I run the localhost version, I cannot click the file button in the navigation bar. Because of this, I'm unable to do any to do any testing on the most recent version of signalserver.
When I make updates in my local repo, the updates push correctly to the remote repo, but the updates are not seen in localhost:8000.
Is there a step missing in the docker instructions? Or maybe an update?
Kelly
QCTools will soon be able to push <streams>
and <format>
data to signalserver.
Here is an example:
<?xml version="1.0" encoding="UTF-8"?>
<!-- Created by QCTools 0.6.0 -->
<ffprobe:ffprobe xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xmlns:ffprobe='http://www.ffmpeg.org/schema/ffprobe' xsi:schemaLocation='http://www.ffmpeg.org/schema/ffprobe ffprobe.xsd'>
<program_version version="git-2017-02-05-e57fd92" copyright="Copyright (c) 2007-2017 the FFmpeg developers" build_date="Feb 8 2017" build_time="20:52:25" compiler_ident="not available" configuration="not available"/>
<library_versions>
<library_version name="libavutil" major="55" minor="46" micro="100" version="3616356" ident="Lavu55.46.100"/>
<library_version name="libavcodec" major="57" minor="75" micro="100" version="3754852" ident="Lavc57.75.100"/>
<library_version name="libavformat" major="57" minor="66" micro="101" version="3752549" ident="Lavf57.66.101"/>
<library_version name="libavfilter" major="6" minor="73" micro="100" version="412004" ident="Lavfi6.73.100"/>
<library_version name="libswscale" major="4" minor="3" micro="101" version="263013" ident="SwS4.3.101"/>
</library_versions>
<frames>
<frame media_type="video" stream_index="0" key_frame="1" pkt_pts="0" pkt_pts_time="0.0000000" pkt_duration_time="0.0333667" pkt_pos="36" pkt_size="337252" width="720" height="486" pix_fmt="yuv422p10le" pict_type="I">
<tag key="lavfi.signalstats.YMIN" value="4"/>
<tag key="lavfi.signalstats.YLOW" value="119"/>
<tag key="lavfi.signalstats.YAVG" value="235.387"/>
<tag key="lavfi.signalstats.YHIGH" value="460"/>
<tag key="lavfi.signalstats.YMAX" value="861"/>
<tag key="lavfi.signalstats.UMIN" value="368"/>
<tag key="lavfi.signalstats.ULOW" value="471"/>
<tag key="lavfi.signalstats.UAVG" value="500.972"/>
<tag key="lavfi.signalstats.UHIGH" value="515"/>
<tag key="lavfi.signalstats.UMAX" value="578"/>
<tag key="lavfi.signalstats.VMIN" value="356"/>
<tag key="lavfi.signalstats.VLOW" value="415"/>
<tag key="lavfi.signalstats.VAVG" value="485.547"/>
<tag key="lavfi.signalstats.VHIGH" value="513"/>
<tag key="lavfi.signalstats.VMAX" value="530"/>
<tag key="lavfi.signalstats.VDIF" value="0"/>
<tag key="lavfi.signalstats.UDIF" value="0"/>
<tag key="lavfi.signalstats.YDIF" value="0"/>
<tag key="lavfi.signalstats.SATMIN" value="0"/>
<tag key="lavfi.signalstats.SATLOW" value="2"/>
<tag key="lavfi.signalstats.SATAVG" value="30.3061"/>
<tag key="lavfi.signalstats.SATHIGH" value="108"/>
<tag key="lavfi.signalstats.SATMAX" value="182"/>
<tag key="lavfi.signalstats.HUEMED" value="44"/>
<tag key="lavfi.signalstats.HUEAVG" value="136.591"/>
<tag key="lavfi.signalstats.TOUT" value="0.00519547"/>
<tag key="lavfi.signalstats.VREP" value="0"/>
<tag key="lavfi.signalstats.BRNG" value="0.0266918"/>
<tag key="lavfi.cropdetect.x1" value="0"/>
<tag key="lavfi.cropdetect.x2" value="720"/>
<tag key="lavfi.cropdetect.y1" value="0"/>
<tag key="lavfi.cropdetect.y2" value="486"/>
<tag key="lavfi.cropdetect.w" value="720"/>
<tag key="lavfi.cropdetect.h" value="486"/>
<tag key="lavfi.psnr.mse.v" value="0"/>
<tag key="lavfi.psnr.mse.u" value="0"/>
<tag key="lavfi.psnr.mse.y" value="0"/>
<tag key="lavfi.psnr.psnr.v" value="0"/>
<tag key="lavfi.psnr.psnr.u" value="0"/>
<tag key="lavfi.psnr.psnr.y" value="0"/>
<tag key="lavfi.ssim.All" value="0"/>
<tag key="lavfi.ssim.V" value="0"/>
<tag key="lavfi.ssim.U" value="0"/>
<tag key="lavfi.ssim.Y" value="0"/>
<tag key="lavfi.idet.single.bff" value="0"/>
<tag key="lavfi.idet.single.tff" value="0"/>
<tag key="lavfi.idet.single.progressive" value="0"/>
<tag key="lavfi.idet.single.undetermined" value="0"/>
<tag key="lavfi.idet.multiple.bff" value="0"/>
<tag key="lavfi.idet.multiple.tff" value="0"/>
<tag key="lavfi.idet.multiple.progressive" value="0"/>
<tag key="lavfi.idet.multiple.undetermined" value="0"/>
<tag key="lavfi.idet.repeated.bottom" value="0"/>
<tag key="lavfi.idet.repeated.top" value="0"/>
<tag key="lavfi.idet.repeated.neither" value="0"/>
<tag key="pkt_duration_time" value="0.0333667"/>
<tag key="pkt_size" value="337252"/>
</frame>
<frame media_type="video" stream_index="0" key_frame="0" pkt_pts="33.3667" pkt_pts_time="0.0333667" pkt_duration_time="0.0333667" pkt_pos="349576" pkt_size="336151" width="720" height="486" pix_fmt="yuv422p10le" pict_type="I">
<tag key="lavfi.signalstats.YMIN" value="4"/>
<tag key="lavfi.signalstats.YLOW" value="119"/>
<tag key="lavfi.signalstats.YAVG" value="235.289"/>
<tag key="lavfi.signalstats.YHIGH" value="459"/>
<tag key="lavfi.signalstats.YMAX" value="874"/>
<tag key="lavfi.signalstats.UMIN" value="376"/>
<tag key="lavfi.signalstats.ULOW" value="471"/>
<tag key="lavfi.signalstats.UAVG" value="500.745"/>
<tag key="lavfi.signalstats.UHIGH" value="515"/>
<tag key="lavfi.signalstats.UMAX" value="579"/>
<tag key="lavfi.signalstats.VMIN" value="358"/>
<tag key="lavfi.signalstats.VLOW" value="416"/>
<tag key="lavfi.signalstats.VAVG" value="485.555"/>
<tag key="lavfi.signalstats.VHIGH" value="513"/>
<tag key="lavfi.signalstats.VMAX" value="539"/>
<tag key="lavfi.signalstats.VDIF" value="2.93949"/>
<tag key="lavfi.signalstats.UDIF" value="3.89837"/>
<tag key="lavfi.signalstats.YDIF" value="7.6502"/>
<tag key="lavfi.signalstats.SATMIN" value="0"/>
<tag key="lavfi.signalstats.SATLOW" value="2"/>
<tag key="lavfi.signalstats.SATAVG" value="30.3544"/>
<tag key="lavfi.signalstats.SATHIGH" value="108"/>
<tag key="lavfi.signalstats.SATMAX" value="175"/>
<tag key="lavfi.signalstats.HUEMED" value="44"/>
<tag key="lavfi.signalstats.HUEAVG" value="132.945"/>
<tag key="lavfi.signalstats.TOUT" value="0.0051126"/>
<tag key="lavfi.signalstats.VREP" value="0"/>
<tag key="lavfi.signalstats.BRNG" value="0.0265947"/>
<tag key="lavfi.cropdetect.x1" value="0"/>
<tag key="lavfi.cropdetect.x2" value="720"/>
<tag key="lavfi.cropdetect.y1" value="0"/>
<tag key="lavfi.cropdetect.y2" value="486"/>
<tag key="lavfi.cropdetect.w" value="720"/>
<tag key="lavfi.cropdetect.h" value="486"/>
<tag key="lavfi.psnr.mse.v" value="0"/>
<tag key="lavfi.psnr.mse.u" value="0"/>
<tag key="lavfi.psnr.mse.y" value="0"/>
<tag key="lavfi.psnr.psnr.v" value="0"/>
<tag key="lavfi.psnr.psnr.u" value="0"/>
<tag key="lavfi.psnr.psnr.y" value="0"/>
<tag key="lavfi.ssim.All" value="0"/>
<tag key="lavfi.ssim.V" value="0"/>
<tag key="lavfi.ssim.U" value="0"/>
<tag key="lavfi.ssim.Y" value="0"/>
<tag key="lavfi.idet.single.bff" value="0"/>
<tag key="lavfi.idet.single.tff" value="0"/>
<tag key="lavfi.idet.single.progressive" value="0"/>
<tag key="lavfi.idet.single.undetermined" value="0"/>
<tag key="lavfi.idet.multiple.bff" value="0"/>
<tag key="lavfi.idet.multiple.tff" value="0"/>
<tag key="lavfi.idet.multiple.progressive" value="0"/>
<tag key="lavfi.idet.multiple.undetermined" value="0"/>
<tag key="lavfi.idet.repeated.bottom" value="0"/>
<tag key="lavfi.idet.repeated.top" value="0"/>
<tag key="lavfi.idet.repeated.neither" value="0"/>
<tag key="pkt_duration_time" value="0.0333667"/>
<tag key="pkt_size" value="336151"/>
</frame>
</frames>
<streams>
<stream index="0" codec_name="ffv1" codec_long_name="FFmpeg video codec #1" codec_type="video" codec_time_base="100/2997" codec_tag_string="FFV1" codec_tag="0x31564646" width="720" height="486" coded_width="720" coded_height="486" has_b_frames="0" sample_aspect_ratio="10:11" display_aspect_ratio="400:297" pix_fmt="yuv422p10le" level="-99" field_order="bt" refs="1" r_frame_rate="2997/100" avg_frame_rate="2997/100" time_base="1/11988" start_pts="0" start_time="0.000000">
<disposition default="1" dub="0" original="0" comment="0" lyrics="0" karaoke="0" forced="0" hearing_impaired="0" visual_impaired="0" clean_effects="0" attached_pic="0" timed_thumbnails="0"/>
<tag key="language" value="eng"/>
<tag key="handler_name" value="DataHandler"/>
<tag key="encoder" value="ffv1"/>
</stream>
<stream index="1" codec_name="pcm_s24le" codec_long_name="PCM signed 24-bit little-endian" codec_type="audio" codec_time_base="1/48000" codec_tag_string="in24" codec_tag="0x34326e69" sample_fmt="s32" sample_rate="48000" channels="2" channel_layout="stereo" bits_per_sample="24" r_frame_rate="0/0" avg_frame_rate="0/0" time_base="1/48000" start_pts="0" start_time="0.000000" bits_per_raw_sample="24">
<disposition default="1" dub="0" original="0" comment="0" lyrics="0" karaoke="0" forced="0" hearing_impaired="0" visual_impaired="0" clean_effects="0" attached_pic="0" timed_thumbnails="0"/>
<tag key="language" value="eng"/>
<tag key="handler_name" value="DataHandler"/>
</stream>
</streams>
<format filename="/Users/davidrice/Desktop/Dave/qctools_dhc_samples/ffv1/DHC0386_Ghosting.mov_ffv1.mov" nb_streams="2" nb_programs="0" format_name="mov,mp4,m4a,3gp,3g2,mj2" format_long_name="QuickTime / MOV" start_time="0.000000" duration="10.678000" size="111637099" bit_rate="83638957" probe_score="100">
<tag key="major_brand" value="qt "/>
<tag key="minor_version" value="512"/>
<tag key="compatible_brands" value="qt "/>
<tag key="encoder" value="Lavf55.13.102"/>
</format>
</ffprobe:ffprobe>
This provides much more info about the file as a whole than we had before (where it had been mostly about frames).
I think we need a design of what data to use and how to present it and then to integrate it into a file view. I suggest these values are relevant to include in display:
With this sample there are line breaks in the resulting report:
6_DHC36.mkv.qctools.xml.gz
But with this sample there are no line breaks:
1A_seattle_parade_transfer_a.qctools.xml.gz
On http://localhost:8000/policy/show/1, I note that each rule has an edit form underneath it but then also includes an 'edit' button, but these 'edit' buttons all just lead back to http://localhost:8000/policy/show/1. So are they needed here?
Presently the dropdowns list frame data from lavfi.cropdetect, lavfi,rebur128, etc, but when new qctools is uploading with new values, the list doesn't update, so it is not possible to make a configuration for metadata from qctools with new values. The filter dropdown on "Add/Delete Filter on Policies" should either update according to supplied data or have some place where the lavfi terms may be registered.
Here is the warning and error:
WARNING: The FILES_VOLUME_PATH variable is not set. Defaulting to a blank string.
Creating volume "signalserver_dbdata" with default driver
Creating volume "signalserver_files" with default driver
ERROR: create .: volume name invalid: "." includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed
The buttons on the Policy page are a little wonky. There are two rename buttons, and they surround the rename field in an odd way. But I'm thinking it's better to just list the policies and their descriptions with an edit button. When they click through to edit, then they can rename, reset, or delete. Kinda like:
Dancing Banana | Flags files with high BRNG metrics | edit
Dave's Party House | Flags files with high saturation levels | edit
Kelly's TOUT Obsession | Self-explanatory | edit
And maybe also include a dropdown menu of all the groups so
Dancing Banana | Flags files with high BRNG metrics | edit | group to process
Thoughts? Bueller?
This is totally up for debate, especially since you guys know better than me about the functionalities of signalserver.
Try as we might the two features of policies below are near impossible for us (me) to understand, let alone use or explain.
Percentage for the Group Process reporting**
Percentage for the File Process reporting***
I am wondering if Ashley or Dave or a savvy third party can re-write the description as if they will be read by people who haven't thought about math since the 5th grade (me).
But I'm also wondering if we can take these features out for now and revisit them at a later date, perhaps after the tester workshop.
?
For instance: http://52.34.188.9:8000/operations/show/Policy%20with%20Spaces gives an error page, but http://52.34.188.9:8000/operations/show/Digibeta_Tests is ok.
When clicking "edit" on any rule, it refreshes the whole page and sets it at the top, it'd be nicer if it just popped up instead!
It returns...
"MultipleObjectsReturned at /videostorage/list/show/test.mkv.qctools
get() returned more than one Video -- it returned 4!"
(etc etc etc)
When editing a rule in a policy, if the user had picked 'average' but then changes it to a something more specific requiring a number, that box does not appear so the user has to edit, change, save, edit again, and then add the number. It should be streamlined so it appears as needed.
Allow create, edit, delete of summarization configurations.
For each summarization allow create, edit, delete of rules of the summarization (such as average of YDIF, or count of frames with BRNG over 77). I'm reminded of the MediaConch policy editor for this.
If running quickstart.sh without declaring FILES_VOLUME_PATH, then the output is like:
WARNING: The FILES_VOLUME_PATH variable is not set. Defaulting to a blank string.
ERROR: Cannot create container for service web: create .: "." includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed
WARNING: The FILES_VOLUME_PATH variable is not set. Defaulting to a blank string.
ERROR: Cannot create container for service web: create .: "." includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed
WARNING: The FILES_VOLUME_PATH variable is not set. Defaulting to a blank string.
ERROR: Cannot create container for service web: create .: "." includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed
WARNING: The FILES_VOLUME_PATH variable is not set. Defaulting to a blank string.
ERROR: Cannot create container for service web: create .: "." includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed
WARNING: The FILES_VOLUME_PATH variable is not set. Defaulting to a blank string.
ERROR: Cannot create container for service web: create .: "." includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed
WARNING: The FILES_VOLUME_PATH variable is not set. Defaulting to a blank string.
ERROR: Cannot create container for service web: create .: "." includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed
Could ./quickstart.sh just use /files
unless the global var is already defined. That way we'd only have to define it if we want to overwrite the default value.
For dark backgrounds!
Suggest adding a section to readme on cleanup and maintenace of docker:
from debugging earlier we used.
docker ps -a -q | xargs docker rm -f
docker images -a -q | wc -l
docker images -a -q | xargs docker rmi -f
Any thoughts about a public policy sharing feature, ala Media Conch?
Nowadays, communication between client and server is sensitive, and a server should accept HTTPS communications, especially if some sensitive information is transported (e.g. a password, users use to use the same password everywhere so scanning the HTTP signalserver communication would permit e.g. an attacker to access the email account of the user).
Related to bavc/qctools#242 (comment)
Show file records in table to allow comparative evaluation.
Sortable per column.
Columns are defined by the applied summarization.
I went to http://localhost:8000/register/
to register an account with an email. Then clicked on http://localhost:8000/password_reset/
to reset the password but get an error:
SMTPSenderRefused at /password_reset/
(530, b'5.5.1 Authentication Required. Learn more at\n5.5.1 https://support.google.com/mail/?p=WantAuthError q3sm8571800qte.0 - gsmtp', '[email protected]')
Request Method: POST
Request URL: http://localhost:8000/password_reset/
Django Version: 1.10.5
Exception Type: SMTPSenderRefused
Exception Value:
(530, b'5.5.1 Authentication Required. Learn more at\n5.5.1 https://support.google.com/mail/?p=WantAuthError q3sm8571800qte.0 - gsmtp', '[email protected]')
Exception Location: /usr/local/lib/python3.4/smtplib.py in sendmail, line 783
Python Executable: /usr/local/bin/python
Python Version: 3.4.5
Python Path:
['/usr/src/app',
'/usr/local/lib/python34.zip',
'/usr/local/lib/python3.4',
'/usr/local/lib/python3.4/plat-linux',
'/usr/local/lib/python3.4/lib-dynload',
'/usr/local/lib/python3.4/site-packages',
'/usr/src/app']
Server time: Thu, 9 Feb 2017 04:05:55 -0500
I'm not sure if this is currently possible, if so let's document it, if not let's implement it.
Hey @yayoiukai
I see fileuploads/templates/base.html and it doesn't seem to be called anywhere -- seems to be the same as the higher-level base.html. Can I remove it?
When I try to delete a file in the File Uploads list page (http://52.34.188.9:8000/fileuploads/list/) I get a "page not found" error message
presently if seems that timestamps are UTC
Either local time of server or of client is fine.
On the policy editor page ("Add/Delete Filter on Policies" ), please alphabetically sort the filter names.
On http://localhost:8000/policy/, each policy provides a text box to rename the policy and then there is a "Rename" button above that box which doesn't seem to do anything, and one rename button to the right of the text box which applies the rename.
Currently the API supports only a password for credentials, and clients have to store the password on their side if they don't want to ask the user to enter the password each time they access the server.
This may be an issue if the computer of the user is compromised, as the password would be retrieved.
A best practice nowadays is to store a token instead of the password on the client side, so the password is transmitted only once and not stored on the client machine (only the token is stored, and can be revoked if the machine is compromised, without providing the password)
Related to bavc/qctools#242 (comment)
Can a report belong to more than one group or is it a 1:1 relationship?
currently it seems xml works but gzipped xml (as made by qctools) don't work
Initially my FILES_VOLUME_PATH values was set to drwxr-xr-x and signalserver produced many errors on upload. I changed to drwxrwxrwx and it worked, but I suggest adding a check somewhere for a minimal permissions on the FILES_VOLUME_PATH
When I run the localhost version, I cannot click the file button in the navigation bar. Because of this, I'm unable to do any to do any testing on the most recent version of signalserver.
Some people might not want to give a description! I don't think this field should be a required field to add a new rule to a policy.
There's a link to the About page, but it has no where to go. Kelly can you draft some text for an "About" page for SignalServer.
I'm not certain in which cases this happens, but I still find instances where the dropdown lists for "Files" and "Account" don't actually drop down, which makes navigation a lot harder.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.