Plan architecture of the asset dumper about assetmanager HOT 20 CLOSED

@Ocramius I like the idea of the warmup. What I'd do for this is allow the user to provide a plan for some tool. This will basically tell the cache warmup which routes to take. This will be a massive project, with a low priority for me personally, because caching will occur just once. I'd like to think this through with you because the idea is something that the asset manager must implement.

from assetmanager.

@RWOverdijk some use cases, such as deploying images on remote CDN servers, require the cache to be already warmed up... That's why it is needed imo...

from assetmanager.

@Ocramius Our CDN actually compiles dynamically. But I do want this feature. I just can't build it at this moment in time as I'm very busy. But I'd love to work it out with you and once fully work out build it. What's your plan? You'll have to supply a map of items to be "warmed up". Because of collections and all that. Maps can be warmed up dynamically but paths can not.

from assetmanager.

@RWOverdijk paths can be scanned recursively looking for file assets...

That would have to be built in each asset resolver separately and just somehow enforced by the resolver interface. Still not sure about the signature :)

from assetmanager.

@Ocramius No, they can't. We're not sure which files in a dir to add, and we're not sure what's actually being used. Also we're not quite sure on the aliases, so paths won't work unless specifically specified by the user.

from assetmanager.

In fact it would be impossible to map dynamic assets (my typical example of something coming from DB).

It is possible though to dump assets for all current existing resolvers by reversing priorities/stacks wherever used.
That way, the system will start from lower priority assets and then overwrite the results when mapping those with higher one...

from assetmanager.

@Ocramius Yes, but that will still not work because we do not know which assets to use. I've thought about this long and hard, and keeping track of an usage array (used for warmup) seems to be the pretties way. (Also has the benefit of knowing what's being used where as you then have a nice overview.).

from assetmanager.

@RWOverdijk I'd assume that a PathStack asset takes all the dir and all subdirs (example).

We can decide that that for each asset resolver, while the aggregate resolver will just iterate and collect return values...

from assetmanager.

I don't think you're following here :p I don't want to flood the cache with assets that will never be used. The fact that assets reside in a dir, doesn't mean that we're actually going to serve it through the asset manager. That's why I think that we do need the map. Obviously the map will only be used for non-mapped assets. So collections and maps are safe.

from assetmanager.

Actually no, ignore what I just said. We really need the uri's. Like my specific use case, for instance, requires me to know the uri to decide where to look. (my asset_manager config is dynamic). So I think we do need an actual map. Perhaps, again, we could create a warmup config, where one of the options will be "massive warmup", which will scan and add everything. Another one should then be a map.

from assetmanager.

@RWOverdijk will think about it... I think we can use filters or dump single resolvers maybe.

from assetmanager.

I've thought about it, and I'll be adding a command line accessible route. You will be able to supply a map (config) that says what should be warmed up. This can also have a callback so building urls will not be as hard. I'll also supply a way to mimic requests, so this can also be done before publishing (why you'd do that is beyond me). Renewing the cache will also be a thing, and this will renew everything in a tmp dir and then cache the assets one by one so there's no high load while renewing or warming up the cache. Expect this, along with some other changes in a v2. (mayor release, lots of changes and additions.)

from assetmanager.

I was just thinking about plain bombing the cache and let LRU come into place if it gets filled :P
Anyway, not sure the issue should be closed ;)

from assetmanager.

I've created a new issue where I'll be adding my plans. Please explain this LRU you speak of. :p

from assetmanager.

LRU cache is basically any cache that deletes entries that are "Last Recently Used" ;)

from assetmanager.

Do you mean "that haven't been used lately"? Because what you're saying
is that it removes the newer entries in stead of the older ones. That
sounds odd :p

On 10/04/2012 04:29 AM, Marco Pivetta wrote:

LRU cache is basically any cache that deletes entries that are "Last
Recently Used" ;)

—
Reply to this email directly or view it on GitHub
#31 (comment).

from assetmanager.

"Last Recently" means "Didn't use it in a while" to me :)
Just FYI: LRU

from assetmanager.

last recently is weird. Also that page says "LEAST recently." But okay :p

On Thu, Oct 4, 2012 at 10:39 AM, Marco Pivetta [email protected]:

"Last Recently" means "Didn't use it in a while" to me :)
Just FYI: LRUhttp://en.wikipedia.org/wiki/Cache_algorithms#Least_Recently_Used

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/31#issuecomment-9134467.

from assetmanager.

Ewww, that's also because of my poor english...
Marco Pivetta

http://twitter.com/Ocramius

http://marco-pivetta.com

On 4 October 2012 10:45, Wesley Overdijk [email protected] wrote:

last recently is weird. Also that page says "LEAST recently." But okay :p

On Thu, Oct 4, 2012 at 10:39 AM, Marco Pivetta [email protected]:

"Last Recently" means "Didn't use it in a while" to me :)
Just FYI: LRU<
http://en.wikipedia.org/wiki/Cache_algorithms#Least_Recently_Used>

—
Reply to this email directly or view it on GitHub<
https://github.com/RWOverdijk/AssetManager/issues/31#issuecomment-9134467>.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/31#issuecomment-9134597.

from assetmanager.

Haha that's fine. Anyway, I'll think about it. Thank you for the
information :)

On Thu, Oct 4, 2012 at 10:49 AM, Marco Pivetta [email protected]:

Ewww, that's also because of my poor english...
Marco Pivetta

http://twitter.com/Ocramius

http://marco-pivetta.com

On 4 October 2012 10:45, Wesley Overdijk [email protected]
wrote:

last recently is weird. Also that page says "LEAST recently." But okay
:p

On Thu, Oct 4, 2012 at 10:39 AM, Marco Pivetta [email protected]:

"Last Recently" means "Didn't use it in a while" to me :)
Just FYI: LRU<
http://en.wikipedia.org/wiki/Cache_algorithms#Least_Recently_Used>

—
Reply to this email directly or view it on GitHub<

https://github.com/RWOverdijk/AssetManager/issues/31#issuecomment-9134467>.

—
Reply to this email directly or view it on GitHub<
https://github.com/RWOverdijk/AssetManager/issues/31#issuecomment-9134597>.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/31#issuecomment-9134679.

from assetmanager.