aws-solutions / live-streaming-on-aws Goto Github PK

The live stream solution is a reference deployment that demonstrates how to deliver highly available live streaming video through an integrated workflow between Elemental Cloud and AWS.

Home Page: https://aws.amazon.com/solutions/implementations/live-streaming-on-aws/

License: Apache License 2.0

Shell 7.24% CSS 0.65% JavaScript 53.10% HTML 2.01% TypeScript 37.01%

ott mediastore aws mediapackage medialive cloudfront live-streaming

live-streaming-on-aws's Introduction

Live Streaming on AWS

How to implement Live streaming on AWS at scale leveraging AWS Elemental MediaLive, MediaPackage and Amazon CloudFront. This repo contains the source code for the AWS solution Live Streaming on AWS.

Architecture Overview

AWS Elemental MediaLive
Is configured to ingest 2 live feeds and transcode the content into multiple adaptive bitrate HLS content. The solution can be configured to ingest RTP RTMP HLS and MediaConnect streams and will apply 1 of 3 encoding profiles which include bitrates of 1080p through 270p. The encoding profile is set at launch and is based on the source resolution (See Encoding Profiles below).

AWS Elemental MediaPackage
Ingests the MediaLive Output and package the Live stream into HLS, DASH, and CMAF formats that are delivered through 3 MediaPackage custom endpoints.

Amazon CloudFront
Is configured with the MediaPackage custom endpoints as the Origins for the distribution. CloudFront then enable the live stream content to be delivered globally and at scale.

Optional Demo Deployment
As part of the CloudFormation template a Demo HTML preview player is deployed to an Amazon S3 bucket which is a single page HTML/JavaScript application that will playback the HLS, DASH, MSS and CMAF streams. In addition, the solution can be configured to ingest a Demo HLS feed hosted on AWS.

Deployment

The solution is deployed using a CloudFormation template with a lambda backed custom resource, available in both NodeJS and Python. For details on deploying the solution please see the details on the Solution home page: Live Streaming on AWS

Encoding Profiles

To solution Configures AWS Elemental MediaLive with one of three encoding profiles based on the source resolution defined at launch as a CloudFormation parameter. The three options are 1080, 720, 540 and correspond to the following encoding profiles:

HD-1080p profile: 1920x1080, 1280x720, 960x540, 768x432, 640x360, 512x288
HD-720p profile: 1280x720, 960x540, 768x432, 640x360, 512x288
SD-540p profile: 960x540, 768x432, 640x360, 512x288

The profiles are defined in JSON and and can be found in:

  source/custom-resource/lib/medialive/encoding-profiles/

Source code

source/custom-resources::
A NodeJS based Lambda function used as a custom resource for deploying MediaLive and MediaPackage resources through CloudFormation.

Deployment

The solution can be deployed through the CloudFormation template available on the solution home page.

Creating a custom build

Prerequisites:

AWS Command Line Interface
Node.js 12.x or later
AWS CDK 1.125.0

The are 2 options for deploying the solution: using the CDK deployment tools or running the build script to generate a CFN template and the packaged lambda code.

Options 1: Deploying through the CDK.

This options simply flollows the standard CDK deployment process. You will need to run cdk bootstrap before you run cdk deploy the first time to setup the cdk resource in your AWS account. Details on using the CDK can be found [here].

Download or clone the repo and make the required changes to the source code.
installl the dependencies for the lambda function and construct.
then run cdk deploy (first time use you will need to run cdk bootstrap first)

cd ./source/custom-resource
npm install --production
cd ../constructs
npm install 
cdk bootstrap //if you havent already done so.
cdk deploy

Option 2: Generate a CloudFormation template.

The CloudFormation template (generated by the CDK) includes a lambda backed custom resource to configure MediaLive and create a UUID. To launch the solution the Lambda source code has to be deployed to an Amazon S3 bucket in the region you intend to deploy the solution.

1. Clone the repo

Download or clone the repo and make the required changes to the source code.

2. Running unit tests for customization

Run unit tests to make sure added customization passes the tests:

cd ./deployment
chmod +x ./run-unit-tests.sh && ./run-unit-tests.sh

3. Create an Amazon S3 Bucket

The CloudFormation template is configured to pull the Lambda deployment packages from Amazon S3 bucket in the region the template is being launched in. Create a bucket in the desired region with the region name appended to the name of the bucket. eg: for us-east-1 create a bucket named: my-bucket-us-east-1

aws s3 mb s3://my-bucket-us-east-1

Ensure that you are owner of the AWS S3 bucket.

aws s3api head-bucket --bucket my-bucket-us-east-1 --expected-bucket-owner YOUR-AWS-ACCOUNT-NUMBER

4. Create the deployment packages

Build the distributable:

chmod +x ./build-s3-dist.sh
./build-s3-dist.sh <my-bucket> live-streaming-on-aws <version>

Notes: The build-s3-dist script expects the bucket name as one of its parameters. This value should not have the region suffix (remove the -us-east-1)

Deploy the distributable to the Amazon S3 bucket in your account:

aws s3 sync ./regional-s3-assets/ s3://my-bucket-us-east-1/live-streaming-on-aws/<version>/ 
aws s3 sync ./global-s3-assets/ s3://my-bucket-us-east-1/live-streaming-on-aws/<version>/

5. Launch the CloudFormation template.

Get the link of the live-streaming-on-aws.template uploaded to your Amazon S3 bucket.
Deploy the solution.

License

This project is licensed under the terms of the Apache 2.0 license. See here LICENSE.

This solution collects anonymized operational metrics to help AWS improve the quality of features of the solution. For more information, including how to disable this capability, please see the implementation guide.

live-streaming-on-aws's People

Contributors

Stargazers

Watchers

Forkers

yaareck nagyistge venkatdoddi gridl hyandell quelves megatiteam junaidpk vothanhkiet jvke paca94 markitodelacruz anas-khan eggoynes turneremeato digitalvideosherpa ginokent evandroviva andreasunterhuber danyal2050 minhlh ab-group arielw10 nktks jelder marcelfahle kkpb dharmsonariya silica-repo ajaynb88 sinclairdigital dormantony ferdinand-007 rajachan piccoloshihan vlodkomr hieu-pv joaquin6 rmcquary mbutan razaphd alandthiessen dejonghe erichigdon raghumeda twinventors dva00jnd sknuwantissera aloysmose jai2033shankar rezabekf disruptive-tech-community tuanva-hw anupammaiti sriharikadali mohsanjaffery ajfayle ritnetsys jfarillas chandrashekar-nallamilli johanbjorn my23 infiniai-tech iammac360 doddipriyambodo ksei1200 tonychue nayan-1992 waqqas-meraj-rt abishek28007 emr-dev1 mayordwells yangaws itechca manthan-rapyder camila0804 vinayasathyanarayana miguelslo27 jayash999 jksessions culy247 creativemindsdev satish5555555 limdongkyu goynese baloda mikereedell palmito357 home-forked lukelan sandimciin trinkdiprovn isabella232 caraveo nukacody sachyav grajzer stephane-monnot craigspaz kibedickson

live-streaming-on-aws's Issues

Rename 'master' branch to 'main'

Changing the non-inclusive term master to main on this release for the main branch.

Pricing: AWS Transfer Family ($.30 per hour - SFTP Endpoint fee)

I have created POC for Live Streaming (Europe Ireland region):

AWS MediaLive
AWS MediaPackage
I have used Standard Definition (SD) (10MBPS) with Single-pipeline channel.

I have used live streaming for max 30-40 minutes. After my testing of live streaming, I had deleted MediaPackage Endpoint and Stopped the MediaLive Channel.

After 24 hours I charged:

MediaLive: $0.89
MediaPackage: $0.04
Transfer Family (AWS Transfer Family EU-ENDPOINT - $.30 per hour - SFTP Endpoint fee: 432.000 Hourly) - $129.60 (?????)

No used:
AWS CloudFront

Why below charges?
Transfer Family (AWS Transfer Family EU-ENDPOINT - $.30 per hour - SFTP Endpoint fee: 432.000 Hourly) - $129.60

Any alternate solution?

[Feature] Add LogsBucket and DemoBucket Logical IDs to Outputs

Is your feature request related to a problem? Please describe.
Outputting LogsBucket and DemoBucket Logical IDs will allow to extend and implement analytics solution https://github.com/aws-samples/amazon-cloudfront-access-logs-queries

Describe the feature you'd like
In the Outputs section of the live-streaming-on-aws.yaml, output LogsBucket and DemoBucket Logical IDs

Outputs:
  LogsBucket:
    Description: Logs bucket
    Value: !Ref LogsBucket
  
  DemoBucket:
    Description: Demo bucket
    Value: !Ref DemoBucket

Thanks,
Franco

Update dependencies node

cdk-nag updates for CDK conversion

We have to add additional meta data for CDK conversion changes to the typescript CDK template file in the constructs directory.

videojs-dash.js file in source has been replaced with a HTML page

The following file: source/console/assets/js/lib/videojs-dash.js currently only contains

<html><body>You are being <a href="https://github-production-release-asset-2e65be.s3.amazonaws.com/38712778/4d55a580-4192-11e9-9a4b-dce60f52f367?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20191211%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20191211T183639Z&X-Amz-Expires=300&X-Amz-Signature=40de12de6c0a64a3a4dfa229c23824b3b8693aa1ad79278cd7180cf392455137&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dvideojs-dash.js&response-content-type=application%2Foctet-stream">redirected</a>.</body></html>

Rather than the source of this code, this seems to be an error, and is failing to load correctly until the asset is replaced with the correct compile version after deployment

Unable to use demo player with RTMP_PUSH configuration

Hey guys,

First of all, I previously installed the template using the URL_PULL config and it worked flawlessly, so thanks!

My problem now: I set up the stack using the RTMP_PUSH config, setting 0.0.0.0/0 as the incoming CIDR block, and the stack deployed successfully.

I open the demo player (Chrome or Firefox, both get the error) to check my RTMP stream.. and I get this error:

Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://abcde.cloudfront.net/out/v1/abcde/index.m3u8. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing).[Learn More]

VIDEOJS: ERROR: (CODE:4 MEDIA_ERR_SRC_NOT_SUPPORTED) The media could not be loaded, either because the server or network failed or because the format is not supported. 
Object { code: 4, message: "The media could not be loaded, either because the server or network failed or because the format is not supported." }

Please let me know if you need any other information.

[Don't Create Medichanel from Cloudformation teamplate]

** Description of the error **
The error could not create a mediachannel with cloudformation tamplate.

** To reproduce **
Execute teamplate run
Enter name stack
Create stack
** Expected behavior **
Create a solution

** Please complete the following information about the solution: **

[] Version: Version 4.0.1
Last updated: 08/2022
Author: AWS
[] Region: US East (N. Virginia)
us-east-1

** Screenshots **
If possible, add a screenshot to help explain your issue (please do NOT include sensitive information).

** Additional context **
After I run the template there was an error lambda couldn't read the SSM paramter store, I fixed this by fixing the teamplate.

"ssm:GetParameters"
"ssm:GetParameter"
After running the template again, I have to wait more than 1 hour, but the mediachannel is still not created.

Input Security Groups not deleted by Delete section

https://github.com/awslabs/live-stream-on-aws/blob/e691332b659652ccf8aff8dddf2e4fd0e833551c/source/custom-resource-py/lambda.py#L81-L98

Does not include code to delete Input Security Groups so these build up as orphan resources in the account as the stack is created / deleted.

Enconding profiles do not support new attribute QvbrQualityLevel for media live

Hi there.
When editing source/custom-resource-py/encoding-profiles/medialive-1080p.json in order to add the new parameter QvbrQualityLevel to VideoDescription node, this script will crash at the moment of creation at CloudFormation with the following error:

Unknown parameter in EncoderSettings.VideoDescriptions[0].CodecSettings.H264Settings: "QvbrQualityLevel", must be one of: AdaptiveQuantization, AfdSignaling, Bitrate, BufFillPct, BufSize, ColorMetadata, EntropyEncoding, FixedAfd, FlickerAq, FramerateControl, FramerateDenominator, FramerateNumerator, GopBReference, GopClosedCadence, GopNumBFrames, GopSize, GopSizeUnits, Level, LookAheadRateControl

Convert custom resources to native AWS for MediaLive & MediaPackage resources (Channels/endpoints)

Describe the feature you'd like
The CloudFormation Template could be updated to use the MediaLive and MediaPackage resource types.

Development notes
Right now we use a custom resource that performs several functions:

creates the MediaLive input channel
can start the channel for the customer if selected
creates the MediaPackage channel
gathers anonymous metrics

Native support for both MediaLive and MediaPackage now exists, so it's preferable to switch to native AWS for both at once. For the switch:

Double check native support for all the endpoints in use
Change out custom resources with native support
Rewrite any customer resources required for functions that are _not_covered by native AWS

Update of template fails

Update of stack properties fails, just editing the URL fails with an error:
CustomResource attribute error: Vendor response doesn't contain Id key in object arn:aws:cloudformation::XXXX ...
(NodeJS env)

Changes for SonarQube findings

Placeholder issue to capture changes for SonarQube findings: bugs, vulnerabilities, and smells.

Adding support for input creation in a VPC

I'm trying to figure out a way of using this in the context of a VPC.
I modified the code to be able to specify a vpc and subnets as required by the SDK documentation, but at creation, the lambda function fails with the following message:

FAILED:: BadRequestException: The Arn of an IAM role that allows MediaLive to manage network interfaces in your VPC is required to create a VPC input.

I've added this to the CustomResourceRole :

  -
    Effect: Allow
    Action:
      - ec2:*
    Resource: 
       - !Join ["", ["arn:aws:ec2:", Ref: "AWS::Region", ":", Ref: "AWS::AccountId", ":*"]]

But it doesn't seem to work.
Any ideas ?

[Question] Is there an easy way of feed the MediaLive with RTSP stream?

I have a stream in RTSP and I would like to feed the MediaLive with this RTSP link, but I couldn't find a way to do this.

Template error while trying to deploy via Cloud Formation template

Hello,
I have done all the steps required to deploy this stack via Cloud Formation, created the bucket that needs to be manually created and everything else. When I try to "Create Stack" via CloudFormation template, I am getting a "CREATE_FAILED" error when the template tries to create a lambda function:

Error occurred while GetObject. S3 Error Code: NoSuchBucket. S3 Error Message: The specified bucket does not exist (Service: AWSLambdaInternal; Status Code: 400; Error Code: InvalidParameterValueException; Request ID: 93428ea8-04cd-11e9-9121-07856613c30c)

This has nothing to do with the bucket that I need to manually create, this has been done. I am aware that it is related to other buckets that the template itself creates. I am also getting a "CREATE_FAILED" error when the template tries to create another bucket, , but what I dont understand is that when I access my S3 console I actually see the additional bucket the template was trying to create and reported failing. It would be awesome if someone could help with the issue.

PS.: when I try to simply visualize the template on "Design", after having copied the template URL into the text box, there is already a syntax error (even before pressing the "Next" button).

Thank you very much,
Marcio

Unable to deploy the 3.1.2 version

Describe the bug
The 3.1.2 seems to have a bug.
I deployed the 3.1.1 solution several times, but for about one week I'm unable to deploy it and it seems to be the 3.1.2 version right now, released a few days ago.

To Reproduce
Deploy the template in us-east-1 or eu-west-1

Expected behavior
A clear and concise description of what you expected to happen.

Please complete the following information about the solution:

Version: v3.1.2
Region: us-east-1, us-west-1
Was the solution modified from the version published on this repository? No
[ ]
If the answer to the previous question was yes, are the changes available on GitHub?
Have you checked your service quotas for the sevices this solution uses?
[x ] Were there any errors in the CloudWatch Logs? No
I have errors on CloudFormation console

CloudFormation did not receive a response from your Custom Resource. Please check your logs for requestId [08c5a1d5-5a28-4736-9e38-b2c35ab3a13c]. If you are using the Python cfn-response module, you may need to update your Lambda function code so that CloudFormation can attach the updated version.

Screenshots
If applicable, add screenshots to help explain your problem (please DO NOT include sensitive information).

Additional context
The deployment is stuck for about one hour and then the error above is displayed then a rollback is triggered.

aws-solutions / live-streaming-on-aws Goto Github PK

live-streaming-on-aws's Introduction

Live Streaming on AWS

Architecture Overview

Deployment

Encoding Profiles

Source code

Deployment

Creating a custom build

Prerequisites:

Options 1: Deploying through the CDK.

Option 2: Generate a CloudFormation template.

1. Clone the repo

2. Running unit tests for customization

3. Create an Amazon S3 Bucket

4. Create the deployment packages

5. Launch the CloudFormation template.

License

live-streaming-on-aws's People

Contributors

Stargazers

Watchers

Forkers

live-streaming-on-aws's Issues

Recommend Projects

Recommend Topics

Recommend Org