scratchfoundation / scratch-render Goto Github PK

Wanted to write down some of the ideas Chris and I discussed about drag-and-drop.

• Since sprites need to be draggable outside of the stage area, they likely need to be rendered outside of the WebGL canvas during a drag.
• One idea is to grab the rendered drawable and paint it to a canvas when the drag begins. The drawable inside the stage can be hidden simultaneously, so it should be transparent to the user. Separately, the VM pauses the scripts on that target.
• The canvas could then be position:fixed/absolute, and dragged anywhere on the webpage (and dropped, for example, into the backpack, or whatever).
• When the drag finishes, the canvas is destroyed and the original drawable is unhidden. The VM should update the position of the sprite. If the drop was inside the stage, move the sprite; otherwise, snap it back to its pre-drag position.

I suspect most of this should be handled by the VM/Scratch-wrapper combo. Things that the renderer will need:

• Ability to hide a drawable (implemented in a PR).
• Ability to render a single drawable to a canvas (or otherwise pull out its pixels and give it to something that can).
• Anything else?

Drawables are drawn in the order they're stored in the list. The current plan is to reshuffle the list when things change layer. What is the plan for the exposed API?

Expected Behavior

Image should not be pixely.

Actual Behavior

Steps to Reproduce

1. In this case, I opened Scratch GUI but I think it is a rendering issue...
2. Observe the already place scratch cat

Operating System and Browser

Chromebook latest version

Moved from scratchfoundation/scratch-vm#405
/cc @CosmicWebServices

When a sprite is in left-right rotation mode, the sprite is rendered with a horizontal flip (rather than a 180-degree rotate). Is this possible in the current renderer, or do we need to add this somehow?

While running in Chrome (Canary, 54) it works fine and at the speed it should. But in Firefox (Nightly, 51), it strongly lags the entire browser. I'm not sure why, but these errors that only appear in Firefox's console might have something to do with it:

Error: WebGL: texImage2D: Incurred CPU-side conversion, which is very slow. (render.js:13794:10)
Error: WebGL: texImage2D: Incurred CPU pixel conversion, which is very slow. (render.js:13794:10)
Error: WebGL: texImage2D: Chosen format/type incurred an expensive reformat: 0x1908/0x1401

I'd test it in Safari as well, but the browser doesn't want to open for me.. :/

Aside from that I'm on a late 2013 iMac running OS X El Capitan 10.10.4.

Steps to Reproduce

• Load VM playground in Microsoft Edge

Reference

SCRIPT5007: Unable to get property 'length' of undefined or null reference
scratch-render.js (11347,1)

if (this._svgDom.children.length < 1 || this._svgDom.children[0].localName !== 'svg') {
    throw new Error('Document does not appear to be SVG.');
}

Operating System and Browser

Windows 10 | Edge (Latest)

Sprites have an origin point (initially the center in Scratch 2.0). Rotations should be applied around this origin.

At the moment (for backwards compatability with 2.0, I assume), pens are rounded. And while I'm not sure what the performance characteristics of canvas are, I can't assume it fairs much better. In particular, in 2.0 putting a single pixel to the screen was a major bottleneck, and IIRC it was because set pen size to 1; pen down produced a circle, not a square.

Might this be configurable with a set pen style to [round v] block?

To make queries consistent across the renderer.

IIRC, all of Scratch's current queries are possible right now, except the one for "sprite touching sprite." Haven't thought about how to implement this myself, but thought it was worth raising.

We've talked about this at some point. Currently, when you scale up a sprite, it gets blurry. I think this is because the SVG rasterization only happens once, while it needs to happen either on resize, or generate multiple sizes at the beginning.

To implement Scratch motion's "keep on stage" functionality, it seems like we may need a way to query a particular drawable's current "actual" (not counting transparent pixels) bounding box. Are there other ways to do this, or should we have a query for this?

We were playing with the color effect and the cat's mouth turned black.

Reported during internal playtest 3/3/17.

Expected Behavior

Sprites should be the same in projects loaded in 2.0 & 3.0.

Actual Behavior

The edges of the sprite are cut-off when you load the project in 3.0.

Steps to Reproduce

Load the example project in 2.0 & 3.0.

• https://scratch.mit.edu/projects/148269787

Operating System and Browser

Mac & Chrome

Moved from scratchfoundation/scratch-gui#194
/cc @jwzimmer

The "balloons" background is a good demo of this.

2.0 reference:

scratch-render:

Note that the balloons look fine in scratch-render, but the "floor" is a solid purple instead of a linear gradient.

Currently this doesn't seem quite right - I think it should be reported as a 1px or 0px square at (x, y).

Reported via internal playtest 3/3/17.

Expected Behavior

Example project where transparency works: https://llk.github.io/scratch-guiLLK/scratch-gui#50919938

Actual Behavior

• Example project where transparency does not work: https://llk.github.io/scratch-guiLLK/scratch-gui#63714086
• rainbow raindrops - transparency not working

Operating System and Browser

Chrome & Mac

Moved from scratchfoundation/scratch-gui#197
/cc @jwzimmer

I'm not 100% sure if this is an issue in the rendering engine or the VM. Filing here for now pending investigation.

Expected Behavior

When loading a project, costumes should be loaded in the rotation and orientation that they were created in.

Actual Behavior

When loading a project, some costumes will be "flipped" horizontally.

Steps to Reproduce

• https://scratch.mit.edu/projects/100736613/ (Original)
• https://llk.github.io/scratch-gui/#100736613 (3.0)

Reference

Operating System and Browser

Mac OS 10.12.2 Chrome Version 56.0.2924.87 (64-bit)

Reported during internal playtest 3/3/17.

Expected Behavior

The color should stay gray scale

Actual Behavior

Changing shade switches color to a shade of red if initial color is gray scale

Operating System and Browser

Mac & Chrome

Moved from scratchfoundation/scratch-gui#193
/cc @jwzimmer

Error

Uncaught TypeError: Cannot read property 'getFastBounds' of undefined
    at http://llk.github.io/scratch-vm/vendor.js:29326:49
    at Array.filter (native)
    at RenderWebGL._filterCandidatesTouching (http://llk.github.io/scratch-vm/vendor.js:29322:42)
    at RenderWebGL.isTouchingDrawables (http://llk.github.io/scratch-vm/vendor.js:29134:34)
    at RenderedTarget.isTouchingSprite (http://llk.github.io/scratch-vm/scratch-vm.js:8457:26)
    at Scratch3SensingBlocks.touchingObject (http://llk.github.io/scratch-vm/scratch-vm.js:15534:28)
    at execute (http://llk.github.io/scratch-vm/scratch-vm.js:16145:30)
    at Sequencer.stepThread (http://llk.github.io/scratch-vm/scratch-vm.js:17188:9)
    at Sequencer.stepThreads (http://llk.github.io/scratch-vm/scratch-vm.js:17146:22)
    at Runtime._step (http://llk.github.io/scratch-vm/scratch-vm.js:16830:38)
    at Runtime.<anonymous> (http://llk.github.io/scratch-vm/scratch-vm.js:17069:14)

Steps to Reproduce

• Load testing project http://llk.github.io/scratch-vm/#141694680
• Press "Start"
• Look at JavaScript console

Operating System and Browser

Mac OS 10.12.2 | Chrome 56.0.2924.76 (64-bit)

When I'm on any site using Scratch Render on my phone, the screen is blinking. It's possible that's because of long refresh time.

System & Browser

Android, Chrome mobile.

Should we include a compiled version in the repo/package?

Not sure if this is too similar to scratchfoundation/scratch-gui#194, in which a sprite was cropped. In this case, a backdrop was cropped or very small in 3.0.

Expected Behavior

Backdrops should be identical when loaded in 2.0 or 3.0 editor.

Actual Behavior

• 54759944
• Project mostly works, but the background/ backdrop is too small
• No console output

Steps to Reproduce

• Compare in 2.0 vs. 3.0: 54759944
• Project mostly works, but the background/ backdrop is too small

Operating System and Browser

Mac & Chrome

Moved from scratchfoundation/scratch-gui#195
/cc @jwzimmer

Repro

• http://llk.github.io/scratch-vm/#133736970
• Play game
• Attempt to blow up an asteroid 🚀 💥

Reference

Uncaught TypeError: Cannot read property 'needsConvexHullPoints' of undefined
    at RenderWebGL.getBounds (scratch-render.js:502)
    at RenderedTarget.getBounds (rendered-target.js:472)
    at RenderedTarget.isTouchingEdge (rendered-target.js:507)
    at Scratch3SensingBlocks.touchingObject (scratch3_sensing.js:40)
    at execute (execute.js:171)
    at Sequencer.stepThread (sequencer.js:109)
    at Sequencer.stepThreads (sequencer.js:67)
    at Runtime._step (runtime.js:601)
    at Runtime.<anonymous> (runtime.js:840)

Originally reported by @SillyInventor as scratchfoundation/scratch-vm#479:

Expected Behavior

Lutin 1, 2, & 3 should have pink images of bubbles, "ok!!!", and "Hazzard!!!" respectively in https://llk.github.io/scratch-gui/#144880441/

Actual Behavior

All missing.

Steps to Reproduce

Navigate to https://llk.github.io/scratch-gui/#144880441/

Operating System and Browser

Ubuntu 16.04, chrome 56.0.2924.87

It would be great to implement the bounding-box optimizations for touching drawable, touching color, and pick. Now that we have basic implementations, we can see that they are showing significant strain on projects:

https://llk.github.io/scratch-vm/#125118090

In Scratch 2.0 this runs much more smoothly than our implementation.

Writing this down so we don't forget. Current renderer treats 100% ghosted sprite as "invisible", but Scratch 2.0 will treat it as if it's there in terms of queries (touching sprite).

Demo:
https://scratch.mit.edu/projects/113427263/

We noticed some strange things about Carl's Cachalot project:
https://llk.github.io/scratch-vm/#73724746

Even though the trees always have y=-10, they sometimes appear very high in the sky! It appears this is the case because there is one costume that has a rotation center in some other coordinate system than the one that's expected (possibly affected by the SVG viewBox). That costume - the one that's positioned incorrectly - is notably generated by Illustrator instead of the Scratch paint editor. :(

A similar thing can be seen with the whale sprite. One of the costumes causes the whale to appear in the top right of the stage. It has a rotation center that is different from the rotation centers of the other two costumes, and that SVG was generated by Illustrator. It's hard to see the whale sprite at first but you can get it if you slow the speed of the project down using the arrow keys.

This code looks possibly suspect:
https://github.com/LLK/scratch-flash/blob/abac440f1f34932679bd001d96de2315be7020ee/src/scratch/ScratchCostume.as#L250

But I wouldn't know where to begin with this.

We may need some plan about how to highlight a drawable when it's clicked in the editor, like this:

It's pretty useful to see which sprite is which.

SVG backdrops are not in the right position after applying rotation center for some reason. Those seem to almost always be off by a few pixels in both X and Y...it looks stage centers are a special case in 2.0: https://github.com/LLK/scratch-flash/blob/master/src/scratch/ScratchObj.as#L219

Example

Correct:

In Scratch:

Browser:

MS Edge/Google Chrome (VM Still Doesn't Work on edge due to default toolbox)

Download:

http://www.wpollock.com/SVG/sphere.svg

• Modifying Playground
• Graphic Effects
• What JS Objects are for and Their Functions

Currently the whirl direction matches Scratch 1.4: clockwise for positive values. In Scratch 2.0 3D, whirl direction is the opposite: counterclockwise for positive values. We suspect we should change to match Scratch 2.0 3D for compatibility (see: #14)

... to match vm and blocks.

Currently, the SVG is redrawn every time the skin is set, which is slow. Perhaps we could cache the draw canvases for future clones?

We need to make a choice at some point who is responsible for these (the "Scratch" wrapper or the renderer itself). One argument for doing it in the renderer is the layering question. In Scratch 1 and 2, monitors can be layered with the other sprites in the project, and they are often used as a UI element (e.g., a score readout).

From a Scratch user:

Looking at the code, my initial guess is that there may be a race condition between creation of the WebGL rendering context and attempting to createBufferFromTypedArray.

Our current pen implementation supports RGBA (albeit in a very hidden way) which is used by many advanced Scratchers.

Example

https://scratch.mit.edu/projects/125372703
https://llk.github.io/scratch-vm/#125372703

For example, try opening project 96384969.

Minimum we need:

• Introduction
• Installation npm install [foobar]
• Getting Started
• Building
• Testing

I've noticed this problem with the VM-renderer combo:

• VM creates a clone of a sprite. A drawable is immediately requested with createDrawable, and a callback is attached.
• Execution of blocks proceeds, possibly for ~2/3*16ms, blocking any response from the callback.
• On return from the callback, the clone immediately sets its properties (x, y, skin, etc.)

Basically, a full frame passes between when we can tell the renderer to make a drawable, and when we can tell it what to put into it. Pretty sure this is at least part of what's leading to lots of squirrels:

That might also be a problem with reloading the costume, not sure.

In Scratch 2.0, these are layered with the sprite itself and positioned according to a bounding box calculated according to what's "actually in" the sprite. E.g., if you start with the cat and run "say Hello", and then edit the cat costume to draw a line in the upper right corner, the say bubble moves:

For show/hide, it would be great to have a separate "visible" flag that determines if a drawable is drawn.

Expected Behavior

Example project: https://scratch.mit.edu/projects/147409812/#player
When ghost effect is applied to a sprite, stamps the sprite with transparency, colors otherwise unchanged.

Actual Behavior

In the art microworld, stamping without changing the ghost effect was fine. After changing the ghost effect with a key stamping did become transparent, but the colors became darker. It looked like it was stamping shadows (grey) instead of the color of the sprite.

I saved the project json as well, but gh won't let me attach it.

Steps to Reproduce

Example project: https://scratch.mit.edu/projects/147409812/#player
Forever
point towards mouse
move 10 steps
change color effect by 10
stamp

When down arrow
change ghost effect by 10

Operating System and Browser

Mac OS 10.11.6 Chrome Version 56.0.2924.87 (64-bit)

Thoughts:

• Provide a method to draw a line from (x1, y1) to (x2, y2) at color (RGB), width (px). VM handles color calculations and deciding when to draw lines.
• Stamp method for current drawable and clear method also provided.

I believe scratch-flash's pen draws the equivalent of canvas lineCap="round". In doing the HTML5 player, I also remember something about drawing lines at .5px off to prevent blur, but there might be a better way to deal with that.

Replace with .editorconfig ala:
https://github.com/LLK/scratch-vm/blob/develop/.editorconfig

Does this sound right:

1. Remove RenderWebGLWorker.js
2. Remove index-webworker.js
3. Remove index-web.js?
4. Remove WorkerMessages.js
5. Remove RenderWebGLLocal.js ? Should we have index.js export RenderWebGL instead?
6. Remove worker handlers here: https://github.com/LLK/scratch-render/blob/develop/src/RenderWebGL.js#L464
7. Remove render-worker.js, render-worker.min.js and references in webpack.config.js