See https://github.com/lbryio/lbry-sdk/releases/tag/v0.44.0

The responses for many of the calls are changing to be a list. Also, account list response changed.

Niko mentioned we are using a setting for the yt downloader that's supposed to get the best qualify video available. A user recently reached out asking what format his videos will be uploaded it and when I did some digging, it looks like we are only grabbing 720p versions.

Example:
https://www.youtube.com/watch?v=NIVms5W0PqQ - has 1080p60 option, but the file we have is in 720p lbry://attack-on-titan-2-review-fight-on#bb202489329ca6f7639e38101c5fc83e265088e7

Some content publishers want to have their videos published for a price.
For this reason ytsync must fetch the following parameters from the API: fee_amount, fee_currency, fee_address and then supply them when calling the publish method.
This issue depends on the implementation on the API side first.

consider the following videos:

1. My awesome dog and its friend speedy (part 1)
2. My awesome dog and its friend speedy (part 2)
3. My awesome dog and its friend speedy (part 3)
4. My awesome dog and its friend speedy (part 4)
5. My awesome dog and its friend speedy (part 5)

the claim name is based on the title, a limit is enforced thus trimming out a part of the name.

claim name for the 5 videos as computed by the algorithm:

1. my-awesome-dog-and-its-friend-sp
2. my-awesome-dog-and-its-friend-sp
3. my-awesome-dog-and-its-friend-sp
4. my-awesome-dog-and-its-friend-sp
5. my-awesome-dog-and-its-friend-sp

lbry.go attempts to publish all videos concurrently. The end result is ambiguous. If they are of similar size they will end up being published in the same block. some will be rejected, some will not. If a video goes through on block 100, the other 4 risk either failing and attempting the very subsequent my-awesome-dog-and-its-friend-sp-1... or updating the previous claim. If they go through in the same block, separate claims with the same claim_name are sent through (I believe).

All in all it's a chaos.
A solution should ensure that no 2 videos can attempt publishing using the same name ever.

My idea: the video tracking table contains the claim names used, said claim names should be returned (and probably are already returned) together with the channel status response payload.
lbry.go needs to check against that list before attempting a publication, if a collision is detected, another name should be attempted.

The codebase needs to be heavily refactored.
This issue wants to track the first stage of refactoring which aims at:

• Using factory pattern to reduce errors/mistakes while instantiating the manager or the sync structs
• bundling data in structures to avoid passing down long lists of variables
• better organizing the code so that it's slightly more readable

dht configs are outdated:
see

use this instead:

SeedNodes: []string{
			"lbrynet1.lbry.com:4444",
			"lbrynet2.lbry.com:4444",
			"lbrynet3.lbry.com:4444",
                        "lbrynet4.lbry.com:4444",
		}

The API can't use floats as they're not reflected into a parseable value.

The solution was to implement it similarly to how all the other types were implemented.

We want to start using the new daemon.

To do that that we need to figure out the API calls that changed and fix them.

Some publishers do not wish to retain copyright or wish to publish under creative commons or similar.

If this information is available via YouTube, we should match it.

If not, we should ask but provide a sane default (e.g. it could be on the lbry.io/youtube/token status page).

Steps:

• When publishing, fetch status.license from youtube api (https://developers.google.com/youtube/v3/docs/videos#status.license). If license is creativeCommon, publish under the creative commons license (check which one youtube uses and use the same). If its youtube, do whatever we do now.
• add a note on the youtube/token page explaining how we'll be setting the license on published content, and tell the user to email us if they want a different treatment for their content. use our support email

Acceptance Criteria

Definition of Done

• Tested against acceptance criteria
• Tested against the assumptions of the user story
• The project builds without errors
• Unit tests are written and passing
• Tests on devices/browsers listed in the issue have passed
• QA performed & issues resolved
• Refactoring completed
• Any configuration or build changes documented
• Documentation updated
• Peer Code Review performed

The UI team would find this beneficial.

Possibly related to #19

When the sync tool pulls an empty queue or finishes its jobs it cycles endlessly spamming slack and the API.

A timeout and a check on the jobs should be added to prevent this.

Currently the default for golang is escape things like <,>, and &. For example > becomes \u003e. We should set this since the json output is not used for html output.

A channel might be in one of the following states:

• pending (awaiting approval by the owner)
• queued (approved, awaiting synchronization)
• syncing (being synchronized by the tool)
• synced (finished synchronizing)
• failed (failure in synchronizing)

The ytsync tool is capable of fetching from all those queues depending on the startup instructions.

When syncing without specific flags the behavior should be the following:

1. process the syncing queue (if anything was left half-way previously it will be finished first)
2. sync the queued queue
3. repeat endlessly or until the maximum number of cycles (specified) is reached

When syncing with the update flag:

1. process the syncing queue (if anything was left half-way previously it will be finished first)
2. sync the synced queue (all previously synced channels will be updated)
3. repeat endlessly or until the maximum number of cycles (specified) is reached

when syncing with a status specified:

1. sync that specified queue only

Something in the logic isn't quite right and the wrong queues are pulled.

See discussion here: rylio/ytdl#45

Is this an issue on our side or something went wrong during the rip/transfer process?

We should determine if we'll ever be syncing larger videos than 2 GB and if so, when would be a good time to process them (probably after adding publish date metadata).

When the sync tool instructs the daemon to shut down, the daemon returns correctly, however it returns too soon and the tool has no clue when the daemon effectively will stop.
For this reason the wallet is being moved before the daemon shuts down and strange things can happen.

In certain cases, not sure if this is related to resolution or video quality, the yt downloader uses a VP80 Codec (https://www.screencast.com/t/YpjByam5) and not the typical web optimized h264. I remember we ran into something similar a while ago with John Cleese videos.

Here's an example: lbry://the-metagame-batman-arkham-asylum-20

Not sure what we can do in this case, but maybe we can force h264 or convert via ffpmeg.

We want to be able to decide sync limits dynamically without having the recompile the binary each time.

For that the following parameters should be added:

• [] Amount of videos to sync per channel at most
• [] Maximum size of a single video to sync (in MB?)

lbry.go is a confusing name for what this does.

Also, should this be public?

When assertions like assert.Nil(t, err) for extras.errors.Err fail in tests, this is the output I'm getting:

            Error Trace:    lbrynet_test.go:75
            Error:          Expected nil, but got: &errors.Error{Err:(*errors.errorString)(0xc0000b06a0), stack:[]uintptr{0x1511f5b, 0x151293a, 0x10fd080, 0x105da11}, frames:[]errors.StackFrame(nil), prefix:""}

In contrast, this is the output for the standard errors.New object in the same spot:

            Error Trace:    lbrynet_test.go:75
            Error:          Expected nil, but got: &errors.errorString{s:"empty structure from sdk client"}

This is not a good choice long-term. Ideally it would be spee.ch.

I would like to write a client program that, given a LBRY URI (e.g. lbry://@irmf#a/irmf-logo-model-1.irmf#4),
the client downloads the data (provided that either the user has already paid for the content or the content is free).

Eventually, it would be nice to allow the user to optionally send credits if the content is gated and they have not already purchased it. But for starting out, I would just like to download the content that is either free or previously-paid-for.

I'm not finding any examples, and searching through the subdirectories, it is not obvious to me how to get started.

If I were to take a random stab-in-the-dark, for example, I might try doing the following:

• lbrycrd.New(lbrycrdURL, chainParams) (What should lbrycrdURL and chainParams be?)

but then I get stuck, as none of the methods appear to be what I want.

Alternatively, maybe I need to go this route:

• blobex.NewBlobExchangeClient(cc) (How should I get cc correctly?)
• blobex.Download(ctx) (How do I specify the URI for the content?)

Can you please provide some guidance on the steps that need to be taken?

Thank you!

Here's the deal: there's a project called "Unified XUL Browser" (also known as "UXP"), which is basically a platform for developing applications using web code. UXP was originally a fork of the Firefox codebase, although there's been significant divergence over the years.

The community that maintains UXP also maintains two browsers that take advantage of it: Basilisk, and Pale Moon. IMHO, they are both a lot lighter than upstream Firefox, which is why I love them. They both use a Gecko-like fork called Goanna.

Unfortunately, if I try to use either browser on beta.lbry.tv, they both give me a blank, white screen. I've checked the url, and it was correct. I've also tried using both browsers with brand new profiles (just in case the problem had something to do with one of my addons), but that didn't help.

Please fix this issue, as the laptop I'm typing this on doesn't *quite*

have enough RAM to use upstream Firefox - much less Chrome, which is an even bigger RAM hog.

Here's a screenshot of what my problem is:

I'm running Ubuntu GNU/Linux x64, if that helps.

I think we may sometimes run into a scenario where we sync a video before it is post-processed in 720P+ on YouTube so we end up syncing a lower quality version. We should check when the video was posted, and if it's very recent (not sure what the timing for post-processing looks like), delay download/publish until the next iteration.

Some channels, particularly those big ones, have duplicates to to previous bugs in the sync process.

A detection method should be implemented, at least temporarily, in the sync process in order to detect and then correct those channels.

Something along this: https://github.com/nikooo777/lbry-duplicate-claims-fixer

Currently ytsync assumes that a channel is first synced and subsequently updated on the same server. This doesn't allow us to balance the load and make full use of fleets (group of temporary servers).

The following steps should be taken:

• Remove redis dependency
• replace current functionalities that use redis with new ones using the new synced_videos table in internal-apis
• remove dependency of the local wallet directory and solely use the S3 storage
• enable versioning of the wallets
• think of an "external" lbrycrd server that feeds the ytsync servers so to avoid having to run a lbrycrd instance for all ytsync servers

In Encoder.Next(), every read creates a blob even if you only read one byte. Instead it should keep reading until it has a full blob's worth of data or hits EOF.

И wилл конверт дем то латин

As per comment here: #10 (comment)

slack logging package should be extracted to own package. Please wait for #10 to be merged first.

Currently, lbry.go checks the sync_server field in order to understand if a given channel can be synced or not (only one server can sync a channel).

By solving this issue both in lbry.go and internal-apis we will be able to update a channel that was previously synced on a different server.

For this to work we need to ensure that two servers can't ever be syncing the same channel at the same time.

*/10 * * * * (/bin/ls /tmp/ | /bin/grep -q ytsync && /usr/bin/find /tmp/ytsync* -mmin +20 -delete) || true
^ the above command might have something to do with this error.

At a certain point the download might fail and from there on all publishes fail.

example: ERRO[62269] error processing video: download error: open /tmp/ytsync259862098/iMvNYejP8xY.mp4: no such file or directory

this is probably also related to what i experienced a couple of days ago

INFO[4719] Processing 9rscUMFPQ8c (14237 in channel)
DEBU[4721] Downloaded 9rscUMFPQ8c
DEBU[4722] Created thumbnail for 9rscUMFPQ8c
DEBU[4722] jsonrpc: publish author=David Pakman Show bid=0.01 channel_name=@davidpakman claim_address=bZxmbHGKxuo5ewDP8Cedkyt5ippCxHHC3i description=--Mitt Romney's mobile app misspells America as "Amercia" and we wonder if that's actually the country that will benefit if Romney becomes President.

--On the Bonus Show: Misleading Drudge headline, TX honor student jailed for missing class, Bilderberg US meeting upcoming, more.

The David Pakman Show is an internationally syndicated talk radio and television program hosted by David Pakman

http://www.davidpakman.com
http://www.davidpakman.com/membership
http://www.davidpakman.com/gear
http://www.facebook.com/davidpakmanshow
...
https://www.youtube.com/watch?v=9rscUMFPQ8c file_path=/tmp/ytsync782624140/9rscUMFPQ8c.mp4 language=en license=Copyrighted (contact author) name=romney-campaign-misspells-america-on thumbnail=https://berk.ninja/thumbnails/9rscUMFPQ8c title=Romney Campaign Misspells AMERICA on Mobile App
INFO[5029] 9rscUMFPQ8c took 5m8.70042584s
ERRO[5030] error processing video: publish error: Post http://localhost:5279: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
INFO[5033] Retrying
INFO[5034] Processing 9rscUMFPQ8c (14237 in channel)
DEBU[5039] 9rscUMFPQ8c already exists at /tmp/ytsync782624140/9rscUMFPQ8c.mp4
DEBU[5040] Downloaded 9rscUMFPQ8c
INFO[5080] 9rscUMFPQ8c took 41.630574618s
ERRO[5085] error processing video: thumbnail error: Put https://jgp4g1qoud.execute-api.us-east-1.amazonaws.com/prod/thumbnail: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
INFO[5089] Retrying
INFO[5090] Processing 9rscUMFPQ8c (14237 in channel)
DEBU[5092] 9rscUMFPQ8c already exists at /tmp/ytsync782624140/9rscUMFPQ8c.mp4
DEBU[5094] Downloaded 9rscUMFPQ8c
INFO[5128] 9rscUMFPQ8c took 37.483586185s
ERRO[5129] error processing video: thumbnail error: Put https://jgp4g1qoud.execute-api.us-east-1.amazonaws.com/prod/thumbnail: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
INFO[5129] Video failed after 3 retries, skipping

so either the download error is not handled correctly or the cronjob deletes videos when it's not supposed to do so.

Acceptance Criteria

Definition of Done

• Tested against acceptance criteria
• Tested against the assumptions of the user story
• The project builds without errors
• Unit tests are written and passing
• Tests on devices/browsers listed in the issue have passed
• QA performed & issues resolved
• Refactoring completed
• Any configuration or build changes documented
• Documentation updated
• Peer Code Review performed

A few dirty things were merged in master to not leave the PR hanging any longer. They should be addressed:

• #10 (comment)
• #10 (comment)
• #10 (comment)
• #10 (comment)
• #10 (comment)
• #10 (comment)
• somewhat #10 (comment)
• #25 (comment)
• eventually discuss #25 (comment) and #25 (comment)
• #25 (comment)
• understand and address #25 (comment)

Rather than publishing the file as the unique id inserted into YouTube, can we instead have a smarter algorithm that trims/compresses the video title?

Something like following replaces coupled with a 25-30 char limit would probably get most of the way:

/[^\p{L}]//
/\s+/-

A smarter version could split on spaces/dashes and try to ensure it only uses a whole number of words.

https://github.com/lbryio/lbry.go/blob/master/jsonrpc/daemon_types.go#L187-L206 needs to be updated according to the new API definition

And if both of them are synced within the same block, a double claim will show up.

Solution:
when ytsync tool detects a time out it should restart the daemon (and thus wait a block too)

Currently the channels are stored using the LBRY desired channel name, however that can conflict. To unify all wallets they must all be renamed using their channelID instead.
this is necessary for #23

Version 17 of lbrycrd deprecated several functions and added others.

We need to find and deprecate all the old ones and replace/add the new ones

In order to be able to finish publishing all UC Berkeley videos to LBRY the following steps need to be taken:

• Adjust cmd/ytsync.go to allow selecting the UCB publisher
• verify that every step of the "download", metadata acquisition, publication, database tracking work
• ensure that redisdb is no longer necessary anywhere in udb (should not)