Giter Club home page Giter Club logo

packages's Introduction

Packages

Source code repository management made simple.

Build Status

Packages is a PHP 5.6 and 7.x application providing an interface and tools for maintaining a private Composer repository. Packages extends Satis, adding a web frontend and useful management functionality like GitHub and GitLab integration.

Packages automatically registers GitLab, GitHub, and Bitbucket project webhooks to keep Satis up to date every time you push code. Packages also features a web management interface that allows for easy management of exposed packages and configured source control repositories.

Packages version 3 works on a plugin based system based around source code repositories. Packages can trigger, with each code push, many automated tasks like documentation generation or code analysis. The simple event-based architecture allows easy creation of new automation tasks.

View the docs online.

Installation

Requirements:

Download the latest release, or clone the repository.

git clone [email protected]:terramar-labs/packages.git

Install dependencies

Switch to the project root directory and run composer install.

cd packages
composer install

Edit configuration

Copy config.yml.dist to config.yml and edit as appropriate.

cp config.yml.dist config.yml
vi config.yml

Generate the database schema

Packages uses Doctrine ORM to auto-generate the database schema for your configured platform.

bin/console orm:schema-tool:create

Running the application

Start PHP's built-in webserver to run the Packages web application with minimal effort.

# Visit http://localhost:8080 to see the landing page.
php -S localhost:8080 -t web

Start a Resque worker

For fully-automatic integration with GitHub, GitLab, and your Satis repository, you must always have at least one Resque worker running.

bin/console resque:worker:start

For more information on Resque workers, check the dedicated section.

Using the application

Read the usage and design documentation for an overview of Packages functionality.

Development/debug mode

Visit index_dev.php in your browser to view the site with the dev environment configuration. In this env, views and the service container are not cached, so changes made are immediately visible.

Customizing

Check out the Contributing Guide for the recommended way to set up your development environment.

Some tips:

  • Views are written using Twig and stored in views/.
    • Views are cached in prod env; use http://localhost:8080/index_dev.php to develop.
    • All pages inherit from views/base.html.twig, except for
    • Public landing page views inherit from views/Default/base.html.twig.
  • Composer Components are used to manage front-end dependencies. The respective web/images/, web/js/bootstrap.min.js, and such are symlinks pointing to the real files installed by Composer in vendor/.
  • Check the documentation for additional information.

Docker support

Packages comes with an example docker-compose.yml that starts an nginx container and a Redis container, ready to get up and running quickly.

Visit the documentation to get started.

Troubleshooting

  1. index_dev.php contains a non-localhost block, comment it out when using Docker.
  2. Manage Resque and Satis using the bin/console command-line utility.
  • Build the Satis packages.json file with the satis:build command.
    bin/console satis:build
  • View queued Resque jobs in Redis with the resque:queue:list command.
    bin/console resque:queue:list
  • View active Resque workers with the resque:worker:list command.
    bin/console resque:worker:list
  • Start a Resque worker with resque:worker:start.
    bin/console resque:worker:start
  1. Check the Resque logs to see if Resque is working properly.
tail -f logs/resque.log

packages's People

Contributors

aeoris avatar ant95 avatar axxon avatar barbarrosa avatar blankse avatar ericrisler avatar frankgiesecke avatar imme-emosol avatar mollux avatar neoblack avatar nickveenhof avatar pavkatar avatar pmacinec avatar taka-oyama avatar tyler-sommer avatar webflo avatar webmaster777 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

packages's Issues

No Metadata Classes to process.

Hello,

I keep getting the following message when I run: bin/console orm:schema-tool:create

No Metadata Classes to process.

What am I doing wrong?

Steps to reproduce:

  • clone the packages repository
  • go inside the repository
  • run composer install
  • copy config.yml.dist to config.yml
  • edit config.yml
  • run bin/console orm:schema-tool:create

I even skipped editing of config.yml and tried to use default values and still the same error.

Best regards,
Vladimir

Create view with Satis-like list of enabled packages

I think it would be good to have open, Satis-like (conceptually, not visually) view with list of available (enabled) packages. It could be available on domain.tld/packages, and it should contain data related to packages.json, but presented in user-friendly way. It should be generated automatically with packages.json.

Enhancement -- GitLab Remote URL expects root URL not remote repository

The language in the Packages Dashboard for adding remotes does not clearly indicate that the remote URL should be the root of the GitLab installation rather than a remote git repository -- which is generally called a "remote" when using git, hence my confusion.

The result of my confusion is that I added a URL that looked like this: http://gitlab.mydomain.com/web-developers/cms.git (not intended to be an actual link)

This returned non-descript HTTP 500 error responses when attempting to Sync, and I had to dig through vendor packages to find out what it was trying to do. When I saw the "/api/packages..." string appended to the end of my URL, I knew how to fix it. In retrospect, this is obvious, but "remote" is what threw me.

Documentation is not strictly clear, but this blog post hints that the root URL is expected: https://tylersommer.com/managing-private-composer-repositories-with-packages

My suggestion is that some kind of detection/validation be done on GitLab URLs when the form is processed to make sure that the URL does not point to a .git repository.

Use jQuery and bootstrap js libs in repo.

Currently, jquery and bootstrap are still loaded from jsdelivr. This should not be the case. Change references to these resources to use the scripts stored under web/js

fatal: I don't handle protocol '[git@https'

HI I am using the packages with GITLAB. Its working great for us. Except there is a small glitch.
When I do
bin/console satis:update --build I am getting an error.

[RuntimeException]                                                                                                                                                                
  Failed to execute git clone --mirror '[git@https://gitlab.myreposerver.com]:company/limited-testing.git' '/root/.cache/composer/vcs/-git-https---gitlab.myreposerver.com--admin-limited-test  
  ing.git/'                                                                                                                                                                         
  Cloning into bare repository '/root/.cache/composer/vcs/-git-https---gitlab.myreposerver.com--admin-limited-testing.git'...                                                            
  fatal: I don't handle protocol '[git@https'  

Now I understand the URL: [git@https://gitlab.myreposerver.com]:company/limited-testing.git is not a valid URL. I tried in changing that in gitlab but the square brackets [] are coming anyways.

Can you help how I can solve this problem ? Many thanks

feature: access to the repository only to members

Hello,

Is there a way to allow only deposited users access to the repository via username and password or a token? If possible, these users / tokens should be managed by means of an API request.

Thanks

Regards
Bjoern

Finish making Sami plugin useful

Currently it just generates docs with default options in a temporary directory.

It should:

  • Be able to clone from a remote (Perhaps another package?)
    or generate a new git repositoriy (and publish?)
  • Generate the documentation
  • Create a new commit
  • Push to configured remote, if any

Other deployment tools can take over from there

Enable private repos

I can't get access to my private Github repos, even when I allow all options in the personal access token. After syncing, I see only my public repos, but no private ones.

Make packages.json and includes private

Any ideas besides adding HTTP_BASIC_AUTH to hide the packages.json and it's respecitve includes from the public?

I'd like to have some Token/Auth based solution but no idea on how to solve this best..

Missing parts in the doc

Hi, I'm missing some information in the docs.

  • What does "Clone this project locally after each code push" mean?
  • If I enable the Composer Satis plugin, how do I install the package?
  • Where can I access the satis packages page which shows the installable versions, etc.?

And one which I was able to figure out so far:

  • Why don't my changes to the twig file show? (hint: delete twig cache in /path...)

Cheers
Markus

Getting 500

Just installed on fresh new server and followed the installation instruction and getting 500 error.

But /index_dev.php is working.

Any insight, how to get started on this packages?

Update composer version

Please,

update the require restriction for package composer/composer, it is in 1.1.2 version yet.

GitLab Webhooks fail

Using GitLab 8.3.2 all webhooks fail with message Hook execution failed: Accepted

Multiple users and user management

reference to #73

Hello Tyler,

thank you for the answer. I have now enabled this feature secure_satis: true and has the problem that has nothing changed.
only after i am clear the folder cacheand satisand run /bin/console satis:build finished the securing for satis.

now composer asks to username and password, how can i now manage this users?
the managed user are not working

And now i see that the /packages URI is protected too.
Can i would like to change it separat, please. Can you change this ?

Thank you,

Regards,

Bjoern

Webhook data validation

When a webhook is triggered, validation should be done using the package's remote.

Implement some data validation service that each remote type can implement.

Something like:

<?php

interface WebHookValidatorInterface
{
    public function validate(Request $request, Package $package);
}

Satis 2.x requires PHP 7+

Currently, Packages still supports PHP 5.6. Composer Satis (relatively) recently started work on 2.x line of Satis and are requiring PHP 7+.

We need to change the requirement in Packages' composer.json to use Satis's 1.x branch:

    "composer/satis": "1.0-dev",

Upgrade Gitlab, Github, Guzzle libraries

Any thoughts on upgrading the Guzzle library to a newer version?

While installing the package I got the notification that: Package guzzle/guzzle is abandoned, you should avoid using it. Use guzzlehttp/guzzle instead.

This can introduce some potential security risks since the old guzzle package is no longer maintained.

Error 500: Sync GitLab Remote - Fatal error: Uncaught PDOException

After click on the Sync Button for my GitLab Remote i got an Error 500.

Fatal error: Uncaught PDOException: SQLSTATE[23000]: Integrity constraint violation: 1451 Cannot delete or update a parent row: a foreign key constraint fails (devpackages.packages_gitlab_configurations, CONSTRAINT FK_1FE0005DF44CABFF FOREIGN KEY (package_id) REFERENCES packages (id)) in /var/www/vhosts/mydomain.de/httpdocs/packages/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php:105 Stack trace: #0 /var/www/vhosts/mydomain.de/httpdocs/packages/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/PDOStatement.php(105): PDOStatement->execute(NULL) #1 /var/www/vhosts/mydomain.de/httpdocs/packages/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php(1006): Doctrine\DBAL\Driver\PDOStatement->execute() #2 /var/www/vhosts/mydomain.de/httpdocs/packages/vendor/doctrine/dbal/lib/Doctrine/DBAL/Connection.php(596): Doctrine\DBAL\Connection->executeUpdate('DELETE FROM pac...', Array, Array) #3 /var/www/vhosts/mydomain.de/htt in /var/www/vhosts/mydomain.de/httpdocs/packages/vendor/doctrine/dbal/lib/Doctrine/DBAL/Driver/AbstractMySQLDriver.php on line 60

bin/console orm:schema-tool:create throws error

I'm not sure whether this is a bug or that I'm doing something wrong.

I installed packages as per http://docs.terramarlabs.com/packages/3.1/getting-started/installation. All goes well, until I run bin/console orm:schema-tool:create. That command throws a MySQL error (or rather, three times the same one):

[Doctrine\DBAL\Exception\ConnectionException]
  An exception occured in driver: SQLSTATE[28000] [1045] Access denied for user 'root'@'localhost' (using password: NO)

I cannot for the life of me figure out why Doctrine tries to connect using 'root'@'localhost' with no password, as I have another with WITH password specified in config.yml. config.yml is in the project root, same location as config.yml.dist

Here are the contents of my config.yml file:

security:
  # Username and password to access the web management interface
  username: ******
  password: ******

doctrine:
  mapping:
    default:
      paths: [ '%app.root_dir%/src/Entity', '%app.root_dir%/src/Plugin' ]
      namespace: Terramar

  database:
    # Any Doctrine driver
    driver: pdo_mysql

    # Options
    ## path: %app.root_dir%/database.sqlite
    host: 127.0.0.1
    user: packages
    password: ******
    dbname: packages

resque:
  host:     'unix:/tmp/redis.sock'
  port:     ~
  database: 0

after install "no database selected" error

After successful install and configuration I got these error in web interface and bin/console orm:schema-tool:create

"SQLSTATE[3D000]: Invalid catalog name: 1046 No database selected"

The database is created and the authentication working and I think the config.yml is correct:

security:
  # Username and password to access the web management interface
  username: admin
  password: xxxx

doctrine:
  mapping:
    default:
      paths: [ '%app.root_dir%/src/Entity', '%app.root_dir%/src/Plugin' ]
      namespace: Terramar

  database:
    # Any Doctrine driver
    driver: pdo_mysql
    host: 127.0.0.1
    database: packages
    user: packages
    password: xxxxxx


resque:
  host:     'unix:/var/run/redis/redis.sock'
  port:     ~
  database: 0

It's not possible to remove a deleted package

I use packages 3.1.0 together with gitlab 8.0.2.
After deleting a repository from gitlab, the package is still present inside packages, also if I sync remotes.
There is no delete function inside packages and if I try to deactivate the package I get an error message:

Fatal error: Uncaught exception 'Gitlab\Exception\RuntimeException' with message '404 Project Not Found' in /var/www/vhosts/domain.tld/packages/vendor/m4tthumphrey/php-gitlab-api/lib/Gitlab/HttpClient/Listener/ErrorListener.php:60 Stack trace: #0 /var/www/vhosts/domain.tld/packages/vendor/m4tthumphrey/php-gitlab-api/lib/Gitlab/HttpClient/HttpClient.php(177): Gitlab\HttpClient\Listener\ErrorListener->postSend(Object(Gitlab\HttpClient\Message\Request), Object(Gitlab\HttpClient\Message\Response)) #1 /var/www/vhosts/domain.tld/packages/vendor/m4tthumphrey/php-gitlab-api/lib/Gitlab/HttpClient/HttpClient.php(109): Gitlab\HttpClient\HttpClient->request('projects/121', Array, 'GET', Array) #2 /var/www/vhosts/domain.tld/packages/vendor/m4tthumphrey/php-gitlab-api/lib/Gitlab/Api/AbstractApi.php(50): Gitlab\HttpClient\HttpClient->get('projects/121', Array, Array) #3 /var/www/ in /var/www/vhosts/domain.tld/packages/vendor/m4tthumphrey/php-gitlab-api/lib/Gitlab/HttpClient/Listener/ErrorListener.php on line 60

Refactor plugin "actions"

Not satisfied with the way actions are implemented. Events might be able to be used instead, but I'm unsure how to link the markup rendering to the domain updating in a coherent way.

Stop resque worker undefined function

Hi I have some trouble with Packages, because it does not update Satis if its triggered by a push or manually by GitLab. It updates only if its triggered in the console.

So I just tried to stop the worker and this is what I get:

[root@x composer]# bin/console resque:worker:stop -a
Stopping x.x.ch:10937:*...
PHP Fatal error:  Call to undefined function Terramar\Packages\Console\Command\Worker\posix_kill() in /var/www/x/src/Console/Command/Worker/StopCommand.php on line 61

Fatal error: Call to undefined function Terramar\Packages\Console\Command\Worker\posix_kill() in /var/www/x/src/Console/Command/Worker/StopCommand.php on line 61

Problem with composer update command

Hello,

after execute the composer install command in to a fresh installation and the execute the command

composer update

i got this error message:

bash-4.1$ composer update
Loading composer repositories with package information
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - composer/satis dev-master requires php ^7.0 -> your PHP version (7.0.27) overridden by "config.platform.php" version (5.6) does not satisfy that requirement.
    - composer/satis dev-master requires php ^7.0 -> your PHP version (7.0.27) overridden by "config.platform.php" version (5.6) does not satisfy that requirement.
    - Installation request for composer/satis dev-master -> satisfiable by composer/satis[dev-master].

i try php 5.6 and 7.0 with the same result.

Regards,

Bjoern

Issue with Design and Forms

Hello,

i used the version 3.1.2 and will now upgrade to 3.2.0 but i have some issues with the new release.
currently i have a issue with the design and all input forms.

image

The Input forms does not working. I can not create a new repository or package.

Regards,

Bjoern

500 error with new sami

You should update your package, if it should be longer a solution for gitlab users ...

I did make a fresh install of the package, but the webhook didnt work anymore, if did a test on the gitlab it will show the following message in the log (and a 500 in the webhook site of gitlab)

Uncaught exception 'Symfony\\Component\\HttpKernel\\Exception\\MethodNotAllowedHttpException' with message 'No route found for "GET /webhook/24/receive": Method Not Allowed (Allow: POST)' in /cluster/var/www/site/packagist/vendor/nice/framework/src/Router/RouterSubscriber.php:63\nStack trace:\n#0 [internal function]: Nice\\Router\\RouterSubscriber->onKernelRequest(Object(Symfony\\Component\\HttpKernel\\Event\\GetResponseEvent), 'kernel.request', Object(Symfony\\Component\\EventDispatcher\\ContainerAwareEventDispatcher))\n#1 /cluster/var/www/site/packagist/vendor/symfony/event-dispatcher/EventDispatcher.php(181): call_user_func(Array, Object(Symfony\\Component\\HttpKernel\\Event\\GetResponseEvent), 'kernel.request', Object(Symfony\\Component\\EventDispatcher\\ContainerAwareEventDispatcher))\n#2 /cluster/var/www/site/packagist/vendor/symfony/event-dispatcher/EventDispatcher.php(46): Symfony\\Component\\EventDispatcher\\EventDispatcher->doDispatch(Array, 'kernel.request', Object(Symfony\\Component\\HttpKernel\\Event\\GetResponseEv in /cluster/var/www/site/packagist/vendor/nice/framework/src/Router/RouterSubscriber.php on line 63

And btw ... sami with dev-master is a very bad idea, because they did update and force now php7 (little bit to earlier in my opinion, its not your fault)

Can you please have a look, because your tool is great, but if the webhook doesnt work, it is senseless, because the bin/console satis:update and satis:build didnt work too anymore.

Enable possibility to add AUTH to redis

Hi

We want to run redis with a password. There doesn't seem to be a way to use the rescue workers with AUTH. How about giving the option in the config to set a password?

Static site generator plugin

Integrate something like jekyll to generate a site with configurable options when code is pushed.

This will function to automatically generate documentation sites (docs exist within code bases) and should function similarly to the Sami plugin (see #6)

Satis Update and Build job not including dist in output

When I run bin/console satis:update --build in the repo I get the this in the output which I expect
"dist": { "type": "zip", "url": "https://gitlab.url.com/api/v3/projects/namespace%2Frepo/repository/archive.zip?sha=hash", "reference": "hash", "shasum": "" },

When it is triggered by the webhook in gitlab that section is always missing but everything else is the same.

Webhook rebuilds everything instead of just the repo you want it to.

Is there any reason why the web hooks cause php bin/console satis:update --build to be run on its own?

Surely it should be running a command to just tell satis to rebuild that individual repository.

As it stands packages ends up being unusable when you've got a lot of repos as it means it's forever performing a full rebuild.

contributing improvements

Hello,

I'd like to contribute some improvements to packages, but I have a big question upfront.

Is there a specific reason not to use symfony all the way?
Would you be happy if I migrate the code to symfony, or should I start a new project and use packages just as a dependency?

The best,
Simon

background tasks not run

hey folks,
thanks a lot for this awesome project.

I realized that our setup does not do any background jobs. worker is running redis is setup. but since I am quite new to redis and resque i dont know howto debug.

Do you have any tips to find out why i have to call bin/console satis:build manually to get a new packages.json generated?

thanks

  • Phil

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.