The current version is 0.17, latest is 0.27 and a lot of improvements has been done js wise, also with this upgrade we reduce the vulnerable packages from ~220 to 1 (tar, in the works).

Finally this also brings some twig helpers and config that makes integrating encore easier.

If seems that Flex bundle doesn't execute this part:

"copy-from-recipe": {
    ...
    "src/": "%SRC_DIR%/"
  }

If that is the manifest content of private recipe.

The content of src/* recipe folder is this:

- src
        - Entity
                - .gitignore
        - Repository
                - .gitignore

This is how to reproduce:
Try to install doctrine original recipe https://github.com/symfony/recipes/blob/master/doctrine/doctrine-bundle/1.6 through symfony-flex-server by composer require doctrine/doctrine-bundle command it will work fine and will copy recipe "src/*" files. But you create private recipe which has exactly the same content as original than composer require doctrine/doctrine-bundle will NOT copy recipe "src/*" files.

My symfony-flex-server configuration is this:

Configuration
-------------

 ----------------------------------- ---------- 
  Proxy for official endpoint         enabled   
  Caching for official endpoint       disabled  
  Local mirror for official recipes   disabled  
  Local mirror for contrib recipes    disabled  
 ----------------------------------- ---------- 

Repo status
-----------

 ---------- --------------------------------------------------- ----------------- ------------ --------------------- 
  Repo       Url                                                 Remote readable   Downloaded   Last Update          
 ---------- --------------------------------------------------- ----------------- ------------ --------------------- 
  private    ...                                                 Yes               Yes          2018-03-30 09:57:13  
  official   https://github.com/symfony/recipes                  Yes               No                                
  contrib    https://github.com/symfony/recipes-contrib          Yes               No                                
 ---------- --------------------------------------------------- ----------------- ------------ --------------------- 

This is the related part of composer install -v content when doctrine recipe is in private repo:

- Configuring doctrine/doctrine-bundle (>=1.6): From private:master
    Enabling the package as a Symfony bundle
    Setting configuration and copying files
    Created "./config/packages/doctrine.yaml"
    Created "./config/packages/prod/doctrine.yaml"
    Added environment variable defaults

as you see it does not create ./src/* files.

as flex is now serverless https://symfony.com/blog/symfony-flex-is-going-serverless there are some changes to the data format:

[LogicException]
  The Flex index is missing a "splits" entry. Did you forget to add "flex:[//defaults](vector://defaults/)" in the "extra.symfony.endpoint" array of your composer.json?

also the official endpoint https://symfony.sh/flex.symfony.com is now gone.

edit:

• https://symfony.com/blog/upgrade-flex-on-your-symfony-projects
• https://symfony.com/blog/symfony-flex-is-going-serverless
• https://www.reddit.com/r/symfony/comments/tavy25/the_flex_server_is_shutting_down_soon_update/
• symfony/flex#890

Hello,

I would like to ask if it is possible to run the server-for-symfony-flex package with Bitbucket Server, this does not work for me

Regards!

The paths of the receipes and maybe other things have change. So the server is not working

@moay why the directory names (private - official - contrib) loaded here ?

The use of utf8_* functions cause json_encode crash when the file is utf8 encoded and PHP is utf8 ready. Please, remove this function, some files have special characteres and it cause error for json_encode.

https://github.com/moay/symfony-flex-server/blob/5dc19cea8215a56ad1fc0ef73ddaf38427cc0dd1/src/Service/Compiler/PackagesCompiler.php#L118

In the next line of code, sometimes the json_encode() function return false due to a malformed UTF8 encoding.
https://github.com/moay/symfony-flex-server/blob/e3de35315172942651164c8a405e37649167a4f6/src/Traits/ProvidesUnescapedJsonResponsesTrait.php#L32

In my case, my recipe contain a favicon.ico file who need to be encoded in UTF8 to avoid this kind of unwanted result.

To fix this problem, I had to convert latin to UTF8 content like:

trait ProvidesUnescapedJsonResponsesTrait
{
    /**
     * @param $data
     * @param int $status
     * @param array $headers
     * @param array $context
     * @return static
     */
    protected function json($data, int $status = 200, array $headers = array(), array $context = array()): JsonResponse
    {
        $json = json_encode($data, JSON_UNESCAPED_SLASHES);
        if (false === $json){
            $data = self::convertFromLatin1ToUtf8Recursively($data);
            $json = json_encode($data, JSON_UNESCAPED_SLASHES);
        }
        return JsonResponse::fromJsonString($json, $status, $headers);
    }

    /**
     * Encode array from latin1 to utf8 recursively
     * @param $dat
     * @see https://stackoverflow.com/a/38398648
     * @return array|bool|string
     */
    public static function convertFromLatin1ToUtf8Recursively($dat)
    {
        if (is_string($dat)) {
            return utf8_encode($dat);
        } elseif (is_array($dat)) {
            $ret = [];
            foreach ($dat as $i => $d) {
                $ret[$i] = self::convertFromLatin1ToUtf8Recursively($d);
            }

            return $ret;
        } elseif (is_object($dat)) {
            foreach ($dat as $i => $d) {
                $dat->$i = self::convertFromLatin1ToUtf8Recursively($d);
            }

            return $dat;
        }

        return $dat;
    }
}

Hope this will help anyone who has the same issue.

Would be great to have some king of hook the trigger recipe update.

So instead of putting the recipes:update in crontab, we would have an url that could be called by a repository webhook.

@moay this is my recipe repo : https://github.com/mahsamnz/demo-recipes

I expect bugloos-api-form.yaml file added to config/routes directory in my base project after composer install but nothing added.

what is the problem? am I write anything wrong?

Hi!
Thanks for this package, I'm using it for set up my own Flex server and it works great.

There is one thing that I'm missing (either it's a missing feature or I'm missing some configuration option): the ability to specify the branch in a private repository that should be used to build recipes.

My current setup:

A single repository (mnocon/recipes) with a branch master. Flex server set to use that repository to serve the recipes.

Issue

The issue I have right now is that I cannot easily test changes made to the recipes - I can only see if I specified them correctly if they are merged to master, but at that point they are already available to all users, which means reverting changes in case of an error. There is no "staging".

What I'd like to have

A single repository (mnocon/recipes) with two branches:

• master (with a Flex server configured to serve recipes from it)
• staging (with another Flex server configured to serve recipes from it)

Then my workflow could look like:

1. merge a change to staging branch
2. test the behaviour using the "staging" flex server
3. if it's working correctly merge the staging branch to master (make it public for everyone).

I'm missing the option to set the branch staging as the source of data for the second Flex server. If I understand correctly this part of the code: https://github.com/moay/server-for-symfony-flex/blob/master/src/RecipeRepo/RecipeRepo.php#L126
the default branch is always used.

My workaround

Two repositories: mnocon/recipes and mnocon/recipes-staging (mnocon/recipes-staging is a fork of mnocon/recipes). For each repository I have a Flex server configured to serve recipes from it.

Then I can achieve my goal with workflow like:

1. merge a change to main branch of mnocon/recipes-staging
2. test the behaviour using the "staging" flex server
3. if it's working correctly perform a merge from recipes-staging to recipes (make it public for everyone).

But in an ideal world I'd like to avoid creating an additional repository.
Please let me know if something is unclear or if I have missed something, I'll be happy to answer any questions.

I haven't tried this yet - but is there support for using an OAuth token to auth to my git repo? We use a github enterprise server in my company and prefer oauth for integrations

Thanks!

CJ

When calling the webhook to update the flex server, I get a 500 error response and the following log file:

[2019-03-16 17:33:34] request.INFO: Matched route "webhook_update". {"route":"webhook_update","route_parameters":{"_route":"webhook_update","_controller":"App\\Controller\\WebhookController::update"},"request_uri":"https://XXX/webhook/update","method":"POST"} []
[2019-03-16 17:33:35] request.CRITICAL: Uncaught PHP Exception Cz\Git\GitException: "The repo url https://[email protected]/XXX.git is not readable" at /www/htdocs/XXX/src/RecipeRepo/RecipeRepo.php line 120 {"exception":"[object] (Cz\\Git\\GitException(code: 0): The repo url https://[email protected]/XXX.git is not readable at /www/htdocs/XXX/src/RecipeRepo/RecipeRepo.php:120)"} []

If I'm running the command via CLI everything works as expected.

I thinkg this could be a user problem. The http server runs with a different user than the CLI user. Is it possible to configure the GIT user/password or ssh key?

contents from post-install.txt added to private recipe does not show during composer require
Example public recipe with post-install.txt: https://github.com/symfony/recipes/tree/master/symfony/framework-bundle/3.3
Have someone tried post-install.txt in private recipe? Maybe I'm doing something wrong.

It seems that in 79b1e4d changes were made in the .gitignore file which lead to a erroneuos state of the commited static assets; most files are missing and the manifest file differs from a real production build as in the mapped filenames do not include a hash.

Due to this there was a Twig error after deploying the latest version, because Twig asset can not find the files listed in the manifest.

This issue actually breaks the latest build.

We added !/public/build/ to the .gitignore file in our private fork, built the assets via npm run build and commited the whole public/build/ directory.

Hi,

I stumbled upon this repository by chance and I think there is an issue with the name. By using Symfony Flex Server, people might think that this is an official Symfony project, which it is not. So, can you please rename the project to something else that does not contain Symfony?

Thank you.

wenn I execute the webhook i get the following error log:

[2019-06-06 14:24:38] request.CRITICAL: Uncaught PHP Exception Symfony\Component\Filesystem\Exception\IOException: "Failed to create symbolic link from "../../../../symfony/framework-bundle/3.3/config/bootstrap.php" to "/app/var/repo/official_backup/behat/symfony2-extension/2.1/config/bootstrap.php"." at /app/vendor/symfony/filesystem/Filesystem.php line 395 {"exception":"[object] (Symfony\Component\Filesystem\Exception\IOException(code: 0): Failed to create symbolic link from "../../../../symfony/framework-bundle/3.3/config/bootstrap.php" to "/app/var/repo/official_backup/behat/symfony2-extension/2.1/config/bootstrap.php". at /app/vendor/symfony/filesystem/Filesystem.php:395)"} []

Seems when there was a switch for the Response class, the change didn't happen for \App\Service\OfficialEndpointProxy.

No tests so it never got caught. Also, this made composer to just fallback into auto generating recipes as it could not reach the official endpoint leaving projects unconfigured. (This took me a while to figure out).

PR incoming.

currently installing packages like symfony/framework-bundle for 4.4 leads to a broken state as symfony recipes now use symlinks for some files and directories.

see https://github.com/symfony/recipes/tree/master/symfony/framework-bundle/4.4

if you access it with the official symfony server everything works like charm but with this server many files are missing. https://flex.symfony.com/p/symfony,framework-bundle,iv4.4.1,1574950347;

In order to improve code quality, we should add PHP-CS-Fixer and automated code style tests. Proposed by @SebTM

i made a fork because i could not get the flex server running on windows. mainly i injected the logger in some services and added log messages.

with the log outputs i fixed flex server to be able to run on windows.

https://gitlab.com/c33s-group/forks/flex-server

Bitbucket URLs differ from github links. This needs to be taken into account when rendering links to recipes.

Sometimes while you setup the server, the repository where you store your private recipes maybe empty. In this case using the web-ui is not possible as "/ui/data" returns a "500" as the Finder-component is throwing an "InvalidArgumentException".

For new Users/running out-of-the box this could be annoying, and the fix is really easy.
Will provide a PR in some minutes.

Symfony Flex sometimes call http://flex.server/p/foo,bar,i4.0.0, that is, omitting the timestamp part, then a warning is dispatched because the array index 3 not exists for $timestamp variable assign.

https://github.com/moay/symfony-flex-server/blob/5dc19cea8215a56ad1fc0ef73ddaf38427cc0dd1/src/Service/Provider/PackagesProvider.php#L150

After running this command

./bin/console recipes:reset

I'm getting following output:

[ERROR] Git error: The repo url https://bitbucket.org/<user>/recipes is not readable

[OK] Official recipes repo resetted.

[OK] Contrib recipes repo resetted.

The only way I've managed to make it work was manually clone my repo into var/repo/private

Also it builds wrong URL if recipes are placed on bitbucket:

https://bitbucket.org/<user>/recipes/tree/master/<user>/<bundle>/1.0.1/

instead of

https://bitbucket.org/<user>/recipes/src/master/<user>/<bundle>/1.0.1/

tree instead of src

In our company we use 3rd party Symfony skeleton (one composer.json file) to create the projects. We want to use our private recipes within it. According to this Symfony Flex server documentation we need to add the extra.symfony.endpoint line to our 3rd party Symofiny composer.json file:

    ...
    "extra": {
        "symfony": {
            "id": "...",
            "allow-contrib": true,
            "endpoint": "http://127.0.0.1:8000/"
        }
    }
    ...

If we do that and if we try to run composer install after that, the automated composer install symfony command is not able to create the Symfony project and this is the output from the terminal:

...
  - Installing symfony/cache (v4.0.6): Loading from cache
  - Installing symfony/framework-bundle (v4.0.6): Loading from cache
  - Installing symfony/yaml (v4.0.6): Loading from cache
  - Installing symfony/dotenv (v4.0.6): Loading from cache
Writing lock file
Generating autoload files
Warning: Accessing 127.0.0.1 over http which is an insecure protocol.
Symfony operations: 4 recipes (e75591840502e826575dba2ec6bef69f)
  - Configuring symfony/flex (>=1.0): From private:master

                              
  [ErrorException]            
  Undefined index: .env.dist  

Our full 3rd party Symfony skeleton composer is absolutely identical to official https://github.com/symfony/skeleton/blob/4.0/composer.json except "secure-http": false and "endpoint": "http://127.0.0.1:8000/" are added. Here is a full 3rd party Symfony composer.json content:

{
    "name": "symfony/skeleton",
    "type": "project",
    "license": "MIT",
    "description": "A minimal Symfony project recommended to create bare bones applications",
    "require": {
        "php": "^7.1.3",
        "ext-iconv": "*",
        "symfony/console": "^4.0",
        "symfony/flex": "^1.0",
        "symfony/framework-bundle": "^4.0",
        "symfony/lts": "^4@dev",
        "symfony/yaml": "^4.0"
    },
    "require-dev": {
        "symfony/dotenv": "^4.0"
    },
    "config": {
        "preferred-install": {
            "*": "dist"
        },
        "sort-packages": true,
        "secure-http": false
    },
    "autoload": {
        "psr-4": {
            "App\\": "src/"
        }
    },
    "autoload-dev": {
        "psr-4": {
            "App\\Tests\\": "tests/"
        }
    },
    "replace": {
        "symfony/polyfill-iconv": "*",
        "symfony/polyfill-php71": "*",
        "symfony/polyfill-php70": "*",
        "symfony/polyfill-php56": "*"
    },
    "scripts": {
        "auto-scripts": [
        ],
        "post-install-cmd": [
            "@auto-scripts"
        ],
        "post-update-cmd": [
            "@auto-scripts"
        ]
    },
    "conflict": {
        "symfony/symfony": "*"
    },
    "extra": {
        "symfony": {
            "id": "01C9Q7VP048YH419WJT1WZSMH5",
            "allow-contrib": false,
            "endpoint": "http://127.0.0.1:8000/"
        }
    }
}

We are not able to install custom Symfony skeleton with composer install by using the private Flex server within it.

Hi there. This is in relation to #11 (comment) and #13

We are using gitlab like @lainosantos but unfortunately we have to also add the Port number:

ssh://[email protected]:5000/user/recipes.git

So our format is different again.

We are getting this error from dashboard.js:

TypeError: "n.exec(...) is null"

It is likely from this line: https://github.com/moay/symfony-flex-server/blob/master/assets/js/dashboard.js#L95

I have setup Symfony Flex server and on client I have disabled packagist. I have added one private package to Flex server and I see on web console that it is available (Flex server can see it). When I execute "composer req fk-login" it is correctly "translated" to flipkodskeleton/standard-login-module. Which proves that client does connect to Flex server and gets package information form server. Unfortunately the client can not install package and I get this error:

[InvalidArgumentException]                                                                                                                                                                 
Could not find a matching version of package flipkodskeleton/standard-login-module. Check the package spelling, 
your version constraint and that the package is available in a stability which matches your minimum-stability (stable).

I get the same message if I try to install some non-private package, for example. "composer require api-admin".

I guess I have miss-configured something, just even after reading the whole docs for dozen of times I can not figure what have I done wrong.

Any help would be appreciate.

Best regards.

@SebTM proposed integration of coveralls.io.

Hi,

First of all thanks for your work.

I've carefully followed each step of the setup guide, and everything seems to work fine but when I install a private package only the composer package is downloaded but not the recipe.

So nothing related to the recipe works (bundle not added, recipe files not copied and so on).

The strange thing is that I can see the recipe in the web interface:

And when I do composer require shared-storage, the package is correctly resolved and downloaded.

But I can see that flex auto generate a recipe and then fails because my bundle need to copy a file in the config/packages to define a parameter:

$>composer require shared-storage
Using version dev-master for test/shared-storage-bundle
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Restricting packages listed in "symfony/symfony" to "4.2.*"
Package operations: 1 install, 0 updates, 0 removals
  - Installing test/shared-storage-bundle (dev-master ca71ae3): Loading from cache
Writing lock file
Generating autoload files
ocramius/package-versions:  Generating version class...
ocramius/package-versions: ...done generating version class
Using "https://symfony-recipes.domain.tld" as the Symfony endpoint
Symfony operations: 1 recipe (627c70caa7d0b7771746fc4c744fb3d7)
  - Configuring test/shared-storage-bundle (>=dev-master): From auto-generated recipe
Executing script cache:clear [KO]
 [KO]
Script cache:clear returned with error code 1
!!
!!  In ArrayNode.php line 228:
!!                                                                                        
!!    The child node "project_namespace" at path "test_shared_storage" must be configured.  
!!                                                                                        
!!
!!
Script @auto-scripts was called via post-update-cmd

Installation failed, reverting ./composer.json to its original content.

My repository is hosted on Bitbucket, I have a self hosted satis with the following configuration:

{
  "name": "Test Repositories",
  "homepage": "https://domain.tld",
  "repositories": [
    { "type": "vcs", "url": "https://bitbucket.org/<domain>/symfonysharedstoragebundle.git" }
  ],
  "require-all": true
}

In the composer.json of my project, I've added:

{
    "repositories": [ { "type": "composer", "url": "https://composer-packages.domain.tld/" } ],
    ....
}

Do you have any idea of what I'm missing?

Thanks

Hello :D

The version 1.0.0 has a bug when the private recipe repo url is a ssh git url. Per exemple, i'm using a private gitlab instance and the private repo url is git@gitlab:symfony/recipes.git.

It's beacause the https://github.com/moay/symfony-flex-server/blob/d07a0ef5371413e6c404d7563342545aeb531c5c/assets/js/dashboard.js#L96 not set the new replaced string value to the var repoUrl.

The console error is:

TypeError: Cannot read property '0' of null
    at je.buildRecipeRepoUrl (dashboard.37bb0847d2441133ec9d.js:11)
    at je.n [as buildRecipeRepoUrl] (dashboard.37bb0847d2441133ec9d.js:1)
    at je.recipeUrl (dashboard.37bb0847d2441133ec9d.js:11)
    at je.n [as recipeUrl] (dashboard.37bb0847d2441133ec9d.js:1)
    at eval (eval at Xs (dashboard.37bb0847d2441133ec9d.js:1), <anonymous>:3:2558)
    at je.ie [as _l] (dashboard.37bb0847d2441133ec9d.js:1)
    at je.eval (eval at Xs (dashboard.37bb0847d2441133ec9d.js:1), <anonymous>:3:1167)
    at je.t._render (dashboard.37bb0847d2441133ec9d.js:6)
    at je.i (dashboard.37bb0847d2441133ec9d.js:1)
    at _r.get (dashboard.37bb0847d2441133ec9d.js:6)

at line https://github.com/moay/symfony-flex-server/blob/d07a0ef5371413e6c404d7563342545aeb531c5c/assets/js/dashboard.js#L100

endpoint https://symfony.sh/

my private endpoint: Available updates are showing for almost all packages

is that repairable at all?