Packagist will not list versions and branches containing invalid composer.json files because Composer users will not be able to install these versions. If this happened to a package you maintain, tag a new release with a fixed composer.json. Don’t edit or replace the existing git tag because many tools and services have caches that expect tags to never change.

If you run into this problem and you aren’t sure what’s wrong then there a few things you can do to figure out what’s going on:

Run “composer validate” in your project

The validate command will analyze your composer.json file and list errors as well as warnings and recommendations for publishing your package. You’ll be able to spot JSON syntax errors, but also issues with the contents of your package definition. The output of this command could look like the following example:

~/projects/private-test2$ composer.phar validate                                                                                                                            
./composer.json is valid for simple usage with composer but has                                                                                                                                                     
strict errors that make it unable to be published as a package:                                                                                                                                                     
See https://getcomposer.org/doc/04-schema.md for details on the schema                                                                                                                                              
description : The property description is required                                                                                                                                                                  
Name "seld/PRivate-test2" does not match the best practice (e.g. lower-cased/with-dashes). We suggest using "seld/p-rivate-test2" instead. As such you will not be able to submit it to Packagist.                  
The version field is present, it is recommended to leave it out if the package is published on Packagist.

Check the Update Log

Private Packagist has a link to a package’s update log on the view package page.

The update log will highlight any errors or warnings encountered while updating the package information.

Delete the version attribute from composer.json

Both of the previous examples show you the most common reason for missing releases: A package containing a hardcoded version number in the JSON file which does not match the git tag.

Usually the version attribute gets added to a composer.json early on in a project and everything works fine. But sooner or later someone tags a new release and forgets to increment the version number in composer.json. When Composer now looks at the Git repository it cannot tell whether the tag (1.3.0 in my example) or the version attribute in composer.json (1.2.0 in my example) is the right number. Either option may lead to problems down the road so Composer entirely ignores the broken tag.

The fix is quite simple, delete the version attribute from your composer.json. Composer has great integration with version control systems like Git, Mercurial and Subversion and there is no need to manually track version numbers in a text file for Composer at all. The field really only exists for special situations where a version control system is not in use.

Interested in your own Composer repository? Head over to https://packagist.com to try Private Packagist for free!

Tagged a new release for Composer and it won’t show up on Packagist? was originally published in Private Packagist on Medium, where people are continuing the conversation by highlighting and responding to this story.

Would you like to use a piece of code in your project which is only available for download as a zip file but you’re managing dependencies with Composer? There are a few options to consider: the package repository type, creating your own Git repository to track the zip file’s state or the artifact repository type.

They all have their own pros and cons and it’s not widely understood how managing multiple versions and updating works with these, so this article aims to give you an overview of the options and which ones to pick in which situation. Scroll to the bottom for a quick summary of pros & cons.

The Package Repository

Composer recognizes a special type of repository: package. A package repository lets you define all the information usually retrieved from packagist.org, a VCS repository and the package’s composer.json. This is the only approach which is designed to work even if the zip file does not contain a composer.json file.

"repositories": [
    {
        "type": "package",
        "package": {
            "name": "old-school/magic",
            "version": "1.2.3",
            "dist": {
                "url": "https://old-school-php.com/magic.zip",
                "type": "zip"
            },
            "autoload": {
                "classmap": ["libs/"]
            }
        }
    }
],
"require": {
    "old-school/magic": "1.2.*"
}

You’ll recognize the options from composer.json like name or autoload but you’ll also have to define dist and optionally source to tell Composer where to download the code from. This is usually added automatically by Packagist or in the case of a VCS repository the repository URL itself is the source.

If you are using Private Packagist you can use the same JSON syntax to define a package using the “Add Package -> Custom Package” option. This way you don’t have to add anything to your composer.json, the package will be available in your Composer repository at https://repo.packagist.com/your-org/ like all other packages!

Multiple package versions in a package repository

The package-type repository allows you to specify multiple packages, or multiple versions of the same package. Simply turn the package definition into an array.

This is particularly useful if you use the same repository definition in multiple projects which may need different versions of your zip file. In Private Packagist this is the way to make sure multiple versions of your custom package show up in the Composer repository you use for all your projects.

"repositories": [
    {
        "type": "package",
        "package": [{
                "name": "old-school/magic",
                "version": "1.2.3",
                "dist": {
                    "url": "https://old-school-php.com/magic.zip",
                    "type": "zip"
                }
            },{
                "name": "old-school/magic",
                "version": "1.2.4",
                "dist": {
                    "url": "https://old-school-php.com/magic4.zip",
                    "type": "zip"
                }
            }
        ]
    }
],
"require": {
    "old-school/magic": "1.2.*"
}

Tracking the Zip Contents in a Git Repository

Instead of adding the metadata for the package to your project’s composer.json you can create a Git repository and commit the contents of the unzipped archive you are trying to use. If the zip file already contains a composer.json just tag the contents with the version number (make sure it matches the number in composer.json, or better yet, delete the version from composer.json). If your zip file came without a composer.json create one yourself.

unzip magic.zip
cd magic
git init .
git remote add origin https://github.com/your-org/old-school-magic
# create/edit composer.json here if needed
git add --all
git commit
git tag 1.2.3
git push --tags origin

Using this approach it’s easier to reuse the file in multiple places because you only have to add the VCS repository URL to each project where you’re trying to use the zip file contents. With Private Packagist this Git repository will automatically show up as a package in all your projects using synchronization.

"repositories": [
    {
        "type": "vcs",
        "url": "https://github.com/your-org/old-school-magic"
    }
],
"require": {
    "old-school/magic": "1.2.*"
}

Adding a new version of the zip file to the tracking Git repository

If a new version of the zip file becomes available you can update the contents of the repository and tag the new release. If it doesn’t come with a composer.json make sure to update it as necessary. You need to take special care in this step to ensure deleted files will actually be deleted from your Git repository.

cd magic
rm -r *
unzip ../magic4.zip
# restore/edit composer.json here if needed
git add --all
git commit
git tag 1.2.4
git push origin

Now you can run composer update on any project using the code and it’ll update its dependency on old-school/magic to the latest version 1.2.4.

The Artifact Repository

If your zip file contains a composer.json you have another alternative available to you: the artifact repository type (Of course you could also repackage your zip file to include a composer.json). The artifact repository let’s you specify a local path to a directory containing any number of zip files. Composer will load metadata from composer.json files in all of the zip files. So you can easily store many different packages and versions of these packages in a directory.

"repositories": [
    {
        "type": "artifact",
        "url": "/srv/artifacts/composer/"
    }
],
"require": {
    "old-school/magic": "1.2.*"
}

If you wish to share this project or work on it together however you need to make sure everyone can access the zip files. So you’ll have to either mount a shared filesystem for these artifacts, come up with a distribution mechanism yourself, or actually commit all the zip files into a directory in your project so the artifact repository points to a relative path in your project.

Summary

Repository type: artifact

Pro
- Only one repository entry in composer.json for many packages
Con
- Requires composer.json in zip files
- Zip files need to be on local path on every machine or you have to commit all zip files into project repository which is not really an option if used in multiple projects

Repository type: package

Pro
- Zip file can be used without modification
Con
- Metadata is copied into every project’s composer.json
- New version requires change in every composer.json referencing the package
- Does not use data in composer.json if zip file already contains one

Custom Package on Private Packagist

Pro
- Configuration like package repository type
- Use zip file without modification
- No copying of metadata, no composer.json changes needed
- Single location to update for a new version — even with many projects
Con
- Does not use data in composer.json if zip file already contains one

Git repository to track zip contents

Pro
- Git tools (e.g. diff, log) can be used to view history of the package
- Only short VCS repository entry in every composer.json using the package (Not needed if you use Private Packagist)
Con
- Complex process to update the Git repo every time a new version is released
- Potentially lots of Git repositories to manage if you use many zip files

Head over to https://packagist.com to try Private Packagist for free!

Custom Package Definitions was originally published in Private Packagist on Medium, where people are continuing the conversation by highlighting and responding to this story.

When we first launched Private Packagist on packagist.com it started as a SaaS offer promising an extremely fast setup and effortless maintenance. We chose to launch Private Packagist as a hosted solution first because Toran Proxy was already available to Composer users for local installation. But our plan had always been to replace Toran Proxy with a much improved on-premises version of Private Packagist. So beginning in March we started offering Private Packagist Enterprise for installation on your own infrastructure.

With Private Packagist Enterprise you get full control over the infrastructure Private Packagist is running on. Most notably this means that you can run it inside your VPN behind your firewall allowing it to interact with your protected source control management systems, e.g. GitHub Enterprise, Bitbucket Server / Stash or self-hosted GitLab.

With the help of Replicated we can offer companies a similarly hassle-free setup and maintenance experience as on our cloud platform. All you need is a linux box matching our specs and you can run the Replicated daemon which will take care of setting up the application and allows you to manage Private Packagist through a web based management dashboard. You can create regular snapshots, update the application with a click and easily restore the application from a backup.

From Synchronization to Integration

At the end of February this year we launched our synchronization feature with Bitbucket and GitLab after only supporting GitHub for the first 3 months. It enables you to keep users, permissions and packages in sync with your private code storage reducing the configuration overhead compared to tools like Satis considerably. Automatically configured webhooks further streamline working with Composer packages.

In the meantime we’ve expanded this synchronization feature for Private Packagist Enterprise. You can set up an integration with GitHub Enterprise, Bitbucket Server / Stash or self-hosted GitLab in addition to their public services. This allows your users to authenticate through these services without the need for additional passwords or permission settings. They automatically gain Composer access to the same code they have access to on your source code management system.

Integration Setup Guides

Since the configuration of these integrations and required OAuth credentials can be a little overwhelming we just published a set of installation guides with plenty of screenshots for these services in our documentation:

• GitHub (Enterprise) Integration Setup Guide
• Bitbucket Integration Setup Guide
• Bitbucket Server (Stash) Integration Setup Guide
• GitLab Integration Setup Guide

Head over to https://packagist.com to try Private Packagist for free!

Private Packagist Enterprise was originally published in Private Packagist on Medium, where people are continuing the conversation by highlighting and responding to this story.

We will be attending a number of conferences over the next months and hope to meet you there! If you’re already using Private Packagist we would love to hear your feedback! If you aren’t using Private Packagist yet, stop by to get a live demo and to get all your questions answered in person! Make sure to say hello to Jordi and me and pick up some of our stickers!

May 27–28 Belgrade, Serbia: PHP Srbija Conference
June 9–10 Portsmouth, UK: PHP South Coast
June 15–16 Potsdam, Germany: Contao Konferenz
June 24 Odessa Ukraine: Odessa PHP Conference
June 29-July 1 Amsterdam, Netherlands: Dutch PHP Conference
July 13–16 Malmö, Sweden: Typo3 Developer Days
July 25–26 New York City, USA: Laracon

Head over to https://packagist.com to try Private Packagist for free!

Meet us at a conference near you! was originally published in Private Packagist on Medium, where people are continuing the conversation by highlighting and responding to this story.

When we launched Private Packagist in December of 2016, it included the minimum feature set we deemed necessary to make it a valuable product to our first customers. But due to Composer’s broad support for version control systems, code hosting platforms and authentication protocols, this meant Private Packagist was immediately compatible with all the same systems and platforms, namely Git, Mercurial or Subversion using SSH access, HTTP Basic Auth over SSL, or their native protocols.

To simplify the initial setup and maintenance of a Private Packagist account, we launched with optional synchronization for GitHub organizations. This integration keeps teams, their members, and access permissions in sync with a matching GitHub organization. So you only need to manage users and permissions in a single place.

Synchronization is now available for Bitbucket and GitLab users!

Please give it a try and send us your feedback, we’d love to better understand if it helps or how it could be made even more useful for you specifically.

As of last week, you can use your Bitbucket.org or GitLab.com user accounts to log into Private Packagist at packagist.com. If you already have an account you can connect these services to your existing account on your profile page.

You can then either create a new organization directly synchronized from a Bitbucket Team or a GitLab Group, or you can enable synchronization for an existing Organization in Settings.

Apart from synchronizing teams, users, and permissions, setting up the integration will simplify the addition of new packages to your Composer repository. When you create a new repository on Bitbucket or GitLab it will be added as a Composer package automatically if it contains a composer.json file. If you’d like to add existing repositories as packages, you can do so with the click of a button on the Packages tab in your organization.

Update web hooks notifying Packagist of new code pushes will be installed in the background, and credentials to access the package will be configured automatically.

Head over to https://packagist.com to try Private Packagist for free!

Bitbucket & GitLab Integration was originally published in Private Packagist on Medium, where people are continuing the conversation by highlighting and responding to this story.

When you first run Composer, you usually install some open-source dependencies from its default package archive packagist.org. Packagist.org is the public repository for all open-source PHP packages. So when you add an open-source dependency to your project Composer fetches its metadata, its description, a list of versions, requirements, etc., from Packagist.org.

Installing packages from Packagist.org

Packagist.org only collects and serves package metadata. Most importantly the names of packages, their respective dependencies and the location of their source code. Once Composer resolved the set of dependencies it writes out the composer.lock file containing all metadata for versions of packages that need to be installed. Composer then downloads every package listed in composer.lock. For each package Composer either downloads a distribution file (zip or tar) or clones the respective version control system (git, hg or svn) defined by the package maintainer. By default it selects a distribution file for every tagged release, you can modify Composer’s preference with the “--prefer-source” and “--prefer-dist” options.

This means that installing open-source packages from Packagist.org relies on the respective package maintainers’ source code hosting to be up and running whenever you run composer install. Most of the time that’s GitHub but it may be any other service or even hosted by the package maintainer themselves. Packagist.org doesn’t handle building archives, storing or distributing the package source code.

Removing the single point of failure for installs

When using Composer with Private Packagist, Composer will store two download locations for every package in your composer.lock: the Private Packagist mirror URL and the original download URL. Composer will download the Private Packagist mirror of distribution files which gives you faster downloads. But more importantly even if GitHub, Bitbucket, GitLab or an open-source maintainers self-hosted version control system are down, composer install can still install your dependencies! So if you rely on composer install in your build process it no longer depends on these services being available. But better yet, since Composer stores both URLs this protects you from any Private Packagist downtime too! There is no longer any single point of failure.

Further, mirroring gives you a copy of all dependencies your production system requires, so even if an open-source maintainer deletes their project you can safely switch to a new package at your own pace, because Composer can still install it from your Private Packagist repository until you remove it, too.

Mirrored Repositories on Private Packagist

When you create a new organization on Private Packagist, it is automatically set up to mirror all your dependencies from the open-source package archive Packagist.org. But you can mirror any number of public or private repositories, e.g. the Drupal package repository or Magento Marketplace (see “Mirroring Magento Marketplace Packages”).

By default packages are automatically mirrored and added to your Private Packagist repository the first time they are accessed through composer update. Automated systems using Private Packagist access tokens cannot mirror new packages to ensure that build processes do not have unintended consequences.

You can configure the mirroring policy on a per-repository basis. For example you can ensure new open-source dependencies are discussed or reviewed before they are manually added by an administrator, making them available to all developers in the organization.

Head over to https://packagist.com to try Private Packagist for free!

Mirroring Composer Packages was originally published in Private Packagist on Medium, where people are continuing the conversation by highlighting and responding to this story.

Mirrored packages show up in your Private Packagist search results, your License Review, and can be installed from your Private Packagist repository with Composer. Their distribution files are cached in your Private Packagist organization to make downloads redundant and faster.

To setup credentials for a repository, “Magento Marketplace” in our example, head to Settings > Manage Credentials in your Private Packagist organization. Afterwards you can add a new mirrored third party repository under Settings as well. From then on packages can be manually or automatically mirrored based on your configuration.

Head over to https://packagist.com to try Private Packagist for free!

Magento Marketplace Walkthrough

In the following section I’ll show you every step of mirroring packages from Magento Marketplace. Start by hitting “Manage Credentials” in your organizations settings. My organization is called “Magento-Demo”.

Next enter a description, I’m going with “Magento Marketplace Credentials”, and select “Magento Marketplace” as the authentication type. The domain “repo.magento.com” will be filled in automatically. Now we need the Magento Marketplace Public and Private Key, we’ll click on “My Access Keys” to get to the Magento Marketplace website.

Make sure you’re on the Magento 2 Access Keys page, and then copy and paste the Private and Public keys into the respective fields on Private Packagist. Once you’re done, hit “Create” on the Private Packagist Credentials page.

With the credentials set up we can now mirror a new Composer repository under Settings in the Private Packagist organization.

Pick a name for the repository, I picked “Magento Marketplace”, enter the URL “https://repo.magento.com” and pick the credentials we just set up for authentication with this repository. You can choose whether you would like packages to be mirrored automatically when someone tries to access them through composer update, or if you would like to add them yourself manually.

Under Packages we can now add a package “From Mirror”.

I’m going to enter just one package for the demo, but you can add as many as you like here. I’m going with “magento/module-catalog”. Make sure to select the “Magento Marketplace” Mirror repository underneath, and hit “Add”.

Private Packagist downloads the package metadata in the background and then notifies us that the package has been initialized, and it’s now accessible through composer update & composer install and shows up on package searches in Private Packagist!

Head over to https://packagist.com to try Private Packagist for free!

Mirroring Magento Marketplace Packages in Private Packagist was originally published in Private Packagist on Medium, where people are continuing the conversation by highlighting and responding to this story.

Open-source Licenses

Open-source licenses come in a lot of flavours like weak and strong copyleft or permissive software licenses. In addition to the great benefit of being able to reuse code, open-source licenses present some challenges to businesses. With a complex project making use of open-source frameworks and libraries you will have to determine which open-source licenses allow you to incorporate which work into your own. This depends on the type of product you are building, whether you are providing a service or shipping code to customers, and the license you pick yourself.

The Open Source Initiative is a great resource to learn more about open-source licenses. They have implemented a review process for open-source licenses so it becomes easier to determine whether a given software license is an open-source license at all.

SPDX: Standardized Open-Source License identifiers

The Software Package Data Exchange (SPDX) curates a list of license identifiers that enable automation around licenses in complex systems made up of large numbers of components. Composer makes use of this list with its composer/spdx-licenses library. Composer warns you if the license in your composer.json “license” key cannot be identified using this library. If you maintain any open-source package, please review your composer.json and ensure that you are using a valid SPDX license identifier to help your users manage their dependencies.

Based on the SPDX identifiers Private Packagist License Review provides a list of all open-source licenses used by packages in your package repository. You can browse packages by license, and see if the licenses for a package changed over time.

If you know of or find packages using Private Packagist License Review, which do not use an SPDX identifier, please get in touch with the maintainers or simply send them a pull-request. Often it’s simply a matter of slightly modifying the identifier. By the way, if your package is dual licensed, please specify an array of licenses in your composer.json instead of hardcoding the word “or” into the string to help automated systems understand your licensing.

Private Packagist License Management Roadmap

This is merely the first tool to help you manage the licenses of your dependencies. We’re planning to expand on this functionality by allowing you to define a set of open-source licenses, to allow or reject for new packages, in order to prevent your developers from accidentally requiring packages with an incompatible license. Once we implement notifications, we’ll make you aware of new licenses that you should review when they are first added to your repository.

Head over to https://packagist.com to try Private Packagist for free!

Private Packagist License Review was originally published in Private Packagist on Medium, where people are continuing the conversation by highlighting and responding to this story.

Composer Is An Open-Source Success

Jordi and I created Composer over 5 years ago and it has taken the PHP community by storm. Over 600 people have contributed directly to improvements of the dependency management tool for PHP and billions of packages have been installed with it.

Together with Composer we launched Packagist.org in 2011 to host all open-source PHP packages. Packagist.org has seen tremendous growth every day since: We currently serve over 120,000 packages for a combined total of 690,000 published package versions.

But The Composer Story For Business Has Been Rocky So Far

The few options available to businesses with private source code who would like to use Composer, all have major limitations and are not very convenient to setup or operate.

• Inline VCS/Git repositories in your composer.json files significantly slow down every composer update and are hard to maintain across projects
• Satis provides the bare minimum functionality to access private source code from Composer, but requires manual setup and significant work to operate reliably.
• Toran Proxy has a simpler setup process and supports caching open-source package archives but doesn’t provide permission management and can’t integrate easily with other products. Toran Proxy still needs to be maintained by customers’ own staff.

Private Packagist Addresses Businesses’ Composer Needs

Private Packagist aims to remove all these hurdles for businesses to finally make working with Composer as convenient as it should be. Being a hosted service, setting up your own Composer package repository on Private Packagist is done with a few clicks. No matter if your private source code is hosted on GitHub, GitLab, Bitbucket, any of their on-premise solutions, or in any other Git, Mercurial, or Subversion repository, Private Packagist can immediately access your code after setting up your credentials to make it available for installation through Composer.

Private Packagist also helps businesses better manage and understand their open-source dependencies. Private Packagist already caches all open-source libraries used in your business’s projects and makes them and their metadata (e.g. their license) visible in your private package repository. We will be adding more features to help you better understand risks and analyze the open-source dependencies your business relies on. Further you can restrict the addition of open-source dependencies so you can thoroughly review projects before they are available for use by your developers.

Private Packagist limits your Composer repository to only those packages actually used within your business which improves the performance of composer operations, increasing your developers productivity. Your packages are available redundantly on Private Packagist and their version control system, so that composer install still works for your developers, continuous integration and deployments even if any individual service is unavailable.

Per-user authentication tokens as well as tokens for continuous integration and deployment systems ensure that you can grant and revoke access without a major headache. Fine grained permission management through teams ensures that you can provide teams in your company access to only those packages they would have access to in your version control system. If you’re using GitHub we can fully synchronize team memberships and package access without any manual interaction.

The Road Forward

With our newest contribution to the striving PHP ecosystem we hope to further improve the PHP developer experience. The maintenance of our open-source projects is a joy but also a burden and we hope Private Packagist can play a key part in supporting our open-source efforts. We are going to provide businesses with functionality matching their unique needs and simultaneously provide them with an opportunity to make the PHP open-source ecosystem more sustainable.

So please, head over to Packagist.com and try out Private Packagist. Please send us your feedback or any questions you have either here or via email to contact@packagist.com. We rely on your input to turn Private Packagist into the perfect fit for your needs.

Introducing Private Packagist was originally published in Private Packagist on Medium, where people are continuing the conversation by highlighting and responding to this story.