draft support for Pickle in Composer #3897

pierrejoye · 2015-04-01T09:11:30Z

Draft support for Pickle (https://github.com/FriendsOfPHP/pickle) in Composer.

Spent more time to dig composer code than actually implement that but at least it works to install exts from within a project.

Example project and extension composer.json:
https://gist.github.com/pierrejoye/1bbc2bdcec5887e60490

If you like to test with more extensions:
git clone myext
cd myext
pickle convert

Add the dep to your project. If you do not want to fork an ext, you can create a dep vs a zip file and use the artifact repository, it should work.

Still many things to solve but at least there is something to play with now

Update:

According to our discussions, it is now a two steps process to install extensions, to make sure users understand that installing extensions may affect many projects:

First, the usual command:
composer install

composer will tell you to run
composer install --install-extensions

if the project has extensions dependencies

stof · 2015-04-01T09:29:38Z

src/Composer/Autoload/ClassLoader.php

@@ -345,7 +345,6 @@ private function findFileWithExtension($class, $ext)
 {
 // PSR-4 lookup
 $logicalPathPsr4 = strtr($class, '\\', DIRECTORY_SEPARATOR) . $ext;
-


should be reverted

…ICKLE_PATH (just to ease dev for now)

…nces

…nce, fix naming

pierrejoye · 2015-04-01T10:40:51Z

Made most of the changes you pointed at, except the tmp dir one and the zip downloader

naderman · 2015-04-01T11:04:12Z

I'm really not sure this is a good idea. We discussed in #2898 (comment) why having an ExtensionInstaller is a bad idea, and outlined how this should be implemented instead.

The ExtensionInstaller installs dependencies globally which is very different from how the rest of composer behaves. It does not follow the per-project behavior of composer. If you use the ExtensionInstaller in one project it impacts all other projects on the same machine too, but you will never be informed that you broke the dependencies of the other projects. This needs to be handled much more explicitly and transparent for the user, as is outlined in the comment I linked to.

pierrejoye · 2015-04-01T11:07:04Z

@naderman huh.
We discussed that intensively and you agreed on that. Composer integration was a must have. And one of the ground reason why I started pickle in the 1st place.

Also keeping in mind that multiple php support is coming. That and FPM Pool allows easy per project extensions managements, or fcgi on windows.

So what do you suggest now?

naderman · 2015-04-01T11:07:46Z

Yes I do want it to be integrated, just in the way outlined in that other PR, not as an extension installer.

stof · 2015-04-01T11:22:26Z

src/Composer/Installer/ExtensionInstaller.php

+ protected $pickle = 'pickle';
+ protected $process;
+ /**
+ * Initializes library installer.


this is not the library installer

pierrejoye · 2015-04-02T00:08:20Z

@naderman Ok, after discussions on IRC, makes sense for now. I will try to post a summary :)

Will update the patch accordingly, this PR was a draft to start this exact discussion, so no worry. I am not sure where to start to skip extension during install step 1 and skip non extension during install step 2, somehow an ExtensionInstaller is still required, but do not have to be called if '--install-extensions' is not used. Or?

stof · 2015-04-14T22:48:32Z

how are available extensions known by Composer ? Does pecl.php.net support the Composer protocol to expose packages ? Does it require a special repository implementation ? Or does it need to wait for pickle-web ?

pierrejoye · 2015-04-15T01:08:22Z

No, it won't. Or I don't see why composer should do it. Mind to enlighten
me?
On Apr 15, 2015 5:41 AM, "Christophe Coevoet" [email protected]
wrote:

@pierrejoye https://github.com/pierrejoye the issue is that it would
force Composer to hardcode a list of extensions provided by the core (and
it could still be wrong in case of package managers bundling them
separately, even though they might use strict deps to prevent this). It
would be better to have core extensions being updated to expose the PHP
version as their own version too.

—
Reply to this email directly or view it on GitHub
#3897 (comment).

pierrejoye · 2015-04-15T05:22:50Z

@stof yes, see http://news.php.net/php.internals/85814

and we need a list anyway. Which one is bundled in which major version. They have to be handled differently.

pierrejoye · 2015-04-15T05:23:24Z

Also it is not composer's job to keep that list nor to deal with but pickle, composer does nothing about extensions but check if they are loaded.

sbuzonas · 2015-04-16T10:54:30Z

How far off would a PECL repository be from the PEAR repository?

pierrejoye · 2015-04-16T10:57:59Z

@slbmeh I do not get the question but let me try to answer based on a guess :)

Pickle does not use pear, at all. It does not need package.xml but can handle package with only package.xml but no composer.json (live conversion).

pierrejoye · 2015-04-16T10:58:42Z

Also, I would love feedback about the patch (not CS pls, I will fix them) and if it is how you like to have it.

I can go with a plugin instead for the meantime, as this is a blocker for now.

sbuzonas · 2015-04-16T11:01:11Z

@pierrejoye sorry I had the context in my head. That was a question building off of @stof's question about discovering extensions in regards to a special repository. I believe the PECL and PEAR repositories are rather similar and may be able to work with the existing PEAR stuff.

sbuzonas · 2015-04-16T11:06:36Z

src/Composer/Installer/ExtensionInstaller.php

+
+ protected function createTempDir()
+ {
+ $lockfile = tempnam($this->cacheDir, 'pickle');


Wouldn't this create a file every time you generate a new directory name?

pierrejoye · 2015-04-16T11:11:30Z

@slbmeh please check what Pickle is, it could be a good start.

And no, we are not going to use pear for that, that's exactly the goal, to do not use pear anymore as it is a pain to use as an extension developer.

sbuzonas · 2015-04-16T11:27:15Z

@pierrejoye I'm aware of Pickle. I'm not suggesting anything about Pickle itself. I thought the question was about how will composer know about extensions. My reference to pear was about https://github.com/composer/composer/blob/master/src/Composer/Repository/PearRepository.php, I don't think it's too far of a stretch to make that work with pecl.php.net so you don't have to duplicate metadata.

stof · 2015-04-16T11:31:45Z

@pierrejoye the PlatformRepository providing the info about the platform itself needs to be able to know the version of installed extensions, otherwise you cannot use the version in the constraint.
so if PHP extensions don't expose their version properly, we would have to add such knowledge in Composer itself to reuse the PHP version

stof · 2015-04-16T11:36:15Z

@slbmeh the PearRepository is a huge performance killer, because the PEAR protocol is just not adapter for Composer needs, required a huge number of requests to fetch the necessary data.
This is why Pirum introduced first-class composer support for the pear repositories it generates, to avoid having to use the PearRepository with them.
If we want to make extensions installable with Composer, the extension registry should expose its metadata in an efficient format for Composer (the best idea being to use the composer protocol, used by Packagist and Satis, as this means that Composer already has the support for loading the metadata without doing another implementation).

pierrejoye · 2015-04-16T16:23:20Z

OK, sorry but still no :)

pecl is by far not the place anymore where ppl releases their extensions.
That is why we need pickle.

And pickleweb (packagist for extensions) instead. Bringing the simplicity
of composer to ext developers when it comes to releases, and a good ext
index for the users.
On Apr 16, 2015 6:27 PM, "Steve Buzonas" [email protected] wrote:

@pierrejoye https://github.com/pierrejoye I'm aware of Pickle. I'm not
suggesting anything about Pickle itself. I thought the question was about
how will composer know about extensions. My reference to pear was about
https://github.com/composer/composer/blob/master/src/Composer/Repository/PearRepository.php,
I don't think it's too far of a stretch to make that work with
pecl.php.net so you don't have to duplicate metadata.

—
Reply to this email directly or view it on GitHub
#3897 (comment).

pierrejoye · 2015-04-23T01:03:45Z

@slbmeh pickle works with pecl.php.net transparently and out of th box, even if a release has no composer.json.

Same for the web hooks on registration or release tag.

@stof pickle ensures that the version is set correctly. It cannot be released if not. I also started a discussion on internals to fix the core extensions versions.

pierrejoye · 2015-04-23T01:11:08Z

@stof Will look at the PlaformInfo part, it could be possible to get it to know the installed extensions. I have to check to do not duplicate the installed version management. Not sure how yet.

Waiting on @Seldaek and @naderman to comment on this patch, where it is done and how, then I will investigate further about these things.

pierrejoye · 2015-04-30T00:06:01Z

A couple of things we support in pickle and not sure how to make them work smoothly when called from composer.

Old releases of an extension may not have composer.json (or will most not have), composer will fail on these release. What pickle does is to check for package(2).xml and convert on the fly to generate a usable composer.json. Absence of package.xml will be supported as well, composer.json being created from the src (README, header, etc). It is already partially done for the convert command.

Ideas for the composer part? Or we may simply require it when used from composer, that will help extensions developers to get to add composer.json.

Seldaek · 2015-04-30T08:52:19Z

Well composer doesn't really need the composer.json if the pickle repo provides the metadata. It is needed however if we want to load a github repo of an ext as a vcs repository in composer, so yes if people add it eventually it'd be great.

pierrejoye · 2015-04-30T11:11:40Z

We will enforce it for any new release. But the question is for older
releases, they have none. If composer called pickle for every ext
dependency directly without checking anything, then it is easy.

If not, and the repo (source, VCS, whatever) does not have composer, it may
fail or I do not see how composer can get all the metas.

It also brings me back to the main point. What do you think about the draft
patch? Where it is hooked and so on? :/ about to open beta for the
pickleweb, would be great if we can get composer supporting it. If not, no
big deal but the gap will get bigger.
On Apr 30, 2015 3:52 PM, "Jordi Boggiano" [email protected] wrote:

Well composer doesn't really need the composer.json if the pickle repo
provides the metadata. It is needed however if we want to load a github
repo of an ext as a vcs repository in composer, so yes if people add it
eventually it'd be great.

—
Reply to this email directly or view it on GitHub
#3897 (comment).

Seldaek · 2015-05-05T10:11:22Z

src/Composer/Installer.php

+ if (!$this->installExtensions && $operation->getPackage()->getType() == 'extension') {
+ $this->io->writeError('<warning>This package has extensions dependencies, run composer install --install-extensons</warning>');
+ continue;
+ }


Should maybe support 'update' operations too

If an extension is loaded it cannot be updated. That's the tricky part. It
is also why we somehow need to be able to run pickle against a different
php than the one running pickle. Or use -n but then... :)
On May 5, 2015 5:11 PM, "Jordi Boggiano" [email protected] wrote:

In src/Composer/Installer.php
#3897 (comment):

@@ -523,6 +525,10 @@ protected function doInstall($localRepo, $installedRepo, $platformRepo, $aliases
foreach ($operations as $operation) {
// collect suggestions
if ('install' === $operation->getJobType()) {

if (!$this->installExtensions && $operation->getPackage()->getType() == 'extension') {

$this->io->writeError('<warning>This package has extensions dependencies, run composer install --install-extensons</warning>');

continue;

}

Should maybe support 'update' operations too

—
Reply to this email directly or view it on GitHub
https://github.com/composer/composer/pull/3897/files#r29656783.

Ah right, this kinda sucks though

flip111 · 2016-03-06T14:38:14Z

any news?

gabrielrcouto · 2016-05-18T01:12:41Z

@Seldaek Do you need any help to test and close this PR?

faizanakram99 · 2022-10-20T13:31:22Z

bump

pierrejoye added 2 commits April 1, 2015 10:18

register ExtensionInstaller

daeaebf

register ExtensionInstaller

1a8ee01

stof reviewed Apr 1, 2015
View reviewed changes

pierrejoye added 7 commits April 1, 2015 12:03

ws

8e850ee

WS

b353c3f

default pickle being present in PATH, allow override using COMPOSER_P…

14b47e0

…ICKLE_PATH (just to ease dev for now)

add pickle version and sanity check, add common process to this insta…

f85d7be

…nces

add pickle version and sanity check, add common process to this insta…

8929c3a

…nce, fix naming

only 'extension' type

5bd5e0a

drop unnessary arguements or properties

567b07d

stof reviewed Apr 1, 2015
View reviewed changes

sbuzonas reviewed Apr 16, 2015
View reviewed changes

Seldaek reviewed May 5, 2015
View reviewed changes

alcohol mentioned this pull request Sep 11, 2015

Help needed to make our PHP modules written in C a composer ready. #4409

Closed

alcohol mentioned this pull request Feb 18, 2016

composer-setup fails to check for required mbstring #4929

Closed

Seldaek added this to the Nice To Have milestone Apr 12, 2016

Seldaek added the Feature label Apr 12, 2016

naderman mentioned this pull request Nov 28, 2020

[WIP] Extension installer #2898

Closed

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

draft support for Pickle in Composer #3897

draft support for Pickle in Composer #3897

pierrejoye commented Apr 1, 2015

stof Apr 1, 2015

pierrejoye commented Apr 1, 2015

naderman commented Apr 1, 2015

pierrejoye commented Apr 1, 2015

naderman commented Apr 1, 2015

stof Apr 1, 2015

pierrejoye commented Apr 2, 2015

stof commented Apr 14, 2015

pierrejoye commented Apr 15, 2015

pierrejoye commented Apr 15, 2015

pierrejoye commented Apr 15, 2015

sbuzonas commented Apr 16, 2015

pierrejoye commented Apr 16, 2015

pierrejoye commented Apr 16, 2015

sbuzonas commented Apr 16, 2015

sbuzonas Apr 16, 2015

pierrejoye commented Apr 16, 2015

sbuzonas commented Apr 16, 2015

stof commented Apr 16, 2015

stof commented Apr 16, 2015

pierrejoye commented Apr 16, 2015

pierrejoye commented Apr 23, 2015

pierrejoye commented Apr 23, 2015

pierrejoye commented Apr 30, 2015

Seldaek commented Apr 30, 2015

pierrejoye commented Apr 30, 2015

Seldaek May 5, 2015

pierrejoye May 5, 2015

Seldaek May 5, 2015 via email

flip111 commented Mar 6, 2016

gabrielrcouto commented May 18, 2016

faizanakram99 commented Oct 20, 2022

draft support for Pickle in Composer #3897

Are you sure you want to change the base?

draft support for Pickle in Composer #3897

Conversation

pierrejoye commented Apr 1, 2015

stof Apr 1, 2015

Choose a reason for hiding this comment

pierrejoye commented Apr 1, 2015

naderman commented Apr 1, 2015

pierrejoye commented Apr 1, 2015

naderman commented Apr 1, 2015

stof Apr 1, 2015

Choose a reason for hiding this comment

pierrejoye commented Apr 2, 2015

stof commented Apr 14, 2015

pierrejoye commented Apr 15, 2015

pierrejoye commented Apr 15, 2015

pierrejoye commented Apr 15, 2015

sbuzonas commented Apr 16, 2015

pierrejoye commented Apr 16, 2015

pierrejoye commented Apr 16, 2015

sbuzonas commented Apr 16, 2015

sbuzonas Apr 16, 2015

Choose a reason for hiding this comment

pierrejoye commented Apr 16, 2015

sbuzonas commented Apr 16, 2015

stof commented Apr 16, 2015

stof commented Apr 16, 2015

pierrejoye commented Apr 16, 2015

pierrejoye commented Apr 23, 2015

pierrejoye commented Apr 23, 2015

pierrejoye commented Apr 30, 2015

Seldaek commented Apr 30, 2015

pierrejoye commented Apr 30, 2015

Seldaek May 5, 2015

Choose a reason for hiding this comment

pierrejoye May 5, 2015

Choose a reason for hiding this comment

Seldaek May 5, 2015 via email

Choose a reason for hiding this comment

flip111 commented Mar 6, 2016

gabrielrcouto commented May 18, 2016

faizanakram99 commented Oct 20, 2022