Schema related changes for: lbryio/lbry-sdk#918

Add the following metadata fields:

• file - a dictionary with info about the file. this will grow eventually. for now it should have:
  • name - the filename. 255 chars max.
  • size - the size of the original file
  • duration - an integer that represents how many seconds long the file is. only applies to files that have a duration, like audio or video file.

In the future, we may want to add hash (a hash of the original file) to that dictionary as well, but we will leave it out for now.

Internal Use

Acceptance Criteria

Definition of Done

• Tested against acceptance criteria
• Tested against the assumptions of the user story
• The project builds without errors
• Unit tests are written and passing
• Tests on devices/browsers listed in the issue have passed
• QA performed & issues resolved
• Refactoring completed
• Any configuration or build changes documented
• Documentation updated
• Peer Code Review performed

media_type is the specific term for the property being provided.

content_type is a legacy term that is used because HTTP uses it.

https://stackoverflow.com/questions/9258108/whats-the-difference-between-mediatype-contenttype-and-mimetype

This repo should meet these guidelines

If creators want to link PayPal, Patreon, SubscribeStar, etc., we should help facilitate this in a way stronger than the custom markup in the about section.

(This ticket needs to be spec'ed before work; this is just an idea.)

Some ideas:

1. Note/Comment/Description (like a permanent comment?)
2. Signature (sign stakes with channels?)

See original discussion in lbryio/lbry-sdk#919

Trying to determine if there's a way to tell tips vs supports at the surface, without a type. Seems like comparing to the claim address is the only way, so we should not need this in the metadata. This breaks currently when the claim is updated, so we need to make sure claim updates keep the same address, but we may still want to pursue the type as a configured attribute.

We should allow publishers to set tags for their uploaded content. Would need to discuss implementation details, for example, how many tags do we allow and how is it stored in the metadata. If we want community overlays to work properly, each tag may need to exist on its own, as opposed to a list of tags.

We should allow users to set a Category when uploading content. For YT sync, we can pull this directly from the youtube api.

Can this be free form, or locked down to a set of categories? If we want to use YT's categories, can find the list here: https://developers.google.com/youtube/v3/docs/videoCategories/list (region specific)

Problem

• I'd like to repost someone's claim using a newer schema that has additional fields, and I want to fill in those fields for the reposted claim
• I'd like to repost a claim while changing some of the fields
• I'd like to repost a claim while adding or removing some tags

Proposed solution

Add two new fields to ClaimReference as follows

message ClaimReference {
    bytes claim_hash = 1;
    optional Claim deletions = 2;
    optional Claim edits = 3;
}

edits and deletions are used to change parts of the referenced claim.

• if deletions includes a field that is not in the referenced claim, ignore it
• if deletions includes a repeated field that is in the reference claim, then any values in deletions should be removed from the referenced claim
• if deletions includes a non-repeated field that is in the reference claim, delete those fields from that claim (the value in the deleted field does not matter. it can be anything except the default value for that field)
• if edits includes fields that the referenced claim does not include, or if it includes repeated fields, treat them as added to the claim
• if edits includes non-repeated fields that are also in the referenced claim, treat them as overwriting the existing fields

Deletions must be applied first, and edits applied after.

The type of the deletions and edits Claims must match the type of the claim that is referenced. if it does not match, the edits will not make sense and downstream apps may ignore them or even ignore the claim as a whole.

Downstream apps should make clear to users which values come from the underlying claim and which come from edits/deletions.

If a referenced claim is updated after it is referenced, the values in deletions and edits may no longer make sense. Downstream apps should highlight to the user any fields that were updated in the referenced claim after being edited/deleted.

Outstanding Questions

• What to do if the referenced claim is updated to change the claim type? This gives the original claim's owner a way to "remove" the edits of the repost while still being reposted.

Corner Cases

• Claim A points to a 3-minute-long video, but the duration in the claim is set to 5 minutes
• I repost claim A and fix the duration to be 3 minutes using the edits field
• The owner of claim A updates it to point to a longer video which is 6 minutes long. They set the duration in the metadata to 6 minutes (which is now correct for the new video)
• My repost still has a duration of 3 minutes, which is now wrong

The idea here is that users can mark certain publishes as assets they use in other content, and it's not meant to be searchable / indexed in channels. Pretty much remove it from claim search and only allow on direct resolve.

From @tzarebczan on September 22, 2017 21:23

The Issue

The need to have a release date tag arose during LBRY YouTube sync discussions with @REILLYSMITH . Oftentimes users will want the need to display videos on a channel according to their release date. Right now, all videos are sorted by publish date, which equates to the block they were published or updated on - this may not always be the desired sort order.

The requirement would be to add a Release Date attribute to uploaded content. If not populated, it should default to publish date (original block date if update later) - open to discussion.

System Configuration

• LBRY Daemon version:
• LBRY App version:
• LBRY Installation ID:
• Operating system:

Anything Else

Screenshots

Copied from original issue: lbryio/lbry-sdk#921

Internal Use

Acceptance Criteria

Definition of Done

• Tested against acceptance criteria
• Tested against the assumptions of the user story
• The project builds without errors
• Unit tests are written and passing
• Tests on devices/browsers listed in the issue have passed
• QA performed & issues resolved
• Refactoring completed
• Any configuration or build changes documented
• Documentation updated
• Peer Code Review performed

It has been brought to our attention that many creators would rather disable comments on their content if we don't have proper moderation in place. This would be done via a channel level setting that would be respected by applications (can't actually prevent it on the comment server / blockchain solution).

I'd also argue we should do this on a per claim basis, but Jeremy said channels are a good place to start.

We already have python, JS, and Go. C++ is our last major language that's missing.

Inspired by @btzr-io and Madiator, who need to differentiate between podcasts, songs, and audiobooks, which can all have the same media_type.

Alternatively, we can consider adding richer metadata on a per media type basis.

Fee amount is specified as a float, but it should be specified as a decimal to avoid floating point errors. We should use simplejson library instead of json library as simplejson's dumps supports dumps of Decimal without losing precision.

There is no native decimal type in protobuf, but using string encoded decimal would be sufficient. We should create a new version of Fee , with amount being specified as a string.

Internal Use

Acceptance Criteria

Definition of Done

• Tested against acceptance criteria
• Tested against the assumptions of the user story
• The project builds without errors
• Unit tests are written and passing
• Tests on devices/browsers listed in the issue have passed
• QA performed & issues resolved
• Refactoring completed
• Any configuration or build changes documented
• Documentation updated
• Peer Code Review performed

Needs a proposal

@REILLYSMITH commented on Thu Nov 23 2017

The Issue

Channels cannot host content from other channels in playlists or on their own channel to share/curate/support fellow channels.

Steps to reproduce

Expected behaviour

A support-type claim that enables a channel to "own" channel claims or content claims that already exist, but the key fee still pays to the original claimant.

AKA playlists at blockchain level.

Actual behaviour

Tell us what happens instead

System Configuration

• LBRY Daemon version:
• LBRY App version:
• LBRY Installation ID:
• Operating system:

Anything Else

Screenshots

https://en.wikipedia.org/wiki/ISO_639-3

Otherwise we can't differentiate between traditional and simplified chinese

Recent changes are add new fields like duration which do not apply to all media types (a spreadsheet does not have a duration).

We could easily end up with thousands of metadata properties when we consider type-specific attributes.

What's the correct pattern for this? If it's to define type-specific metadata structures, then it may not be proper to add duration at the top-level for all items.

if we use different versions of protoc, we will constantly have meaningless differences in the generated code

otherwise we'll forget to run build.sh after each change

if committing generated code is not easy, at least error when build should have been run but wasnt

The blockchain will be adding the ability to assign a value to supports. This will allow us store metadata along with supports, including channel signing. The type should be flexible to support multiple use cases.

Use case ideas:
Reposting content(lbryio/lbry-sdk#2028)
Commenting (with/without tips)
Ratings
Community tagging
Blockchain based subscriptions

@lyoshenka commented on Tue Jan 23 2018

@lyoshenka commented on Mon Sep 17 2018

moving to types repo since this repo will probably disappear before this happens

At a minimum:

1. Name
2. Thumbnail
3. Cover
4. Description

Other considerations:

1. Primary Language
2. Featured video
3. Other channels
4. Playlists?

Internal Use

Acceptance Criteria

Definition of Done

• Tested against acceptance criteria
• Tested against the assumptions of the user story
• The project builds without errors
• Unit tests are written and passing
• Tests on devices/browsers listed in the issue have passed
• QA performed & issues resolved
• Refactoring completed
• Any configuration or build changes documented
• Documentation updated
• Peer Code Review performed

We can use that to allow users to select between qualities.

An epic and multiple issues have been proposed for this. There is also a repo lbryio/types that was created for this work.
This epic serves as a meant to tie them all together for completion and prioritization. The issues may need to be updated and refined

The Problem
Some types/objects/classes in LBRY are used across many repos/langauges/apps, and they need to be the same in all these places. They need to be easy to update as they change.

List of Types
claim
transaction
metadata
Proposed Solution
Document the type definitions in a central repo (lbryschema) in a way that can be compiled to language-specific definitions.

lbrynet cannot directly import the generated python code. I think the problem is that protobuf generates files with absolute import paths, and py3 needs them to be relative so they can be imported.

protobuf issue: protocolbuffers/protobuf#1491

our hacky solution: https://github.com/lbryio/lbry/blob/master/lbrynet/schema/Makefile

This is only necessary if we want to use types repo as a module. An alternative is to copy-paste the proto files into lbrynet repo, and then generate the protobufs there.

this description is used for web metadata, rather than visually displayed

it should potentially be called something other than short_description

You may want to use https://github.com/netproteus/protobuf2flowtype, which may require https://github.com/dcodeIO/ProtoBuf.js/ and changes to build.sh

Acceptance Criteria

Definition of Done

• Tested against acceptance criteria
• Tested against the assumptions of the user story
• The project builds without errors
• Unit tests are written and passing
• Tests on devices/browsers listed in the issue have passed
• QA performed & issues resolved
• Refactoring completed
• Any configuration or build changes documented
• Documentation updated
• Peer Code Review performed