There already exists a couple different PPAs which host the latest versions of Redis, however, they pose a significant security threat.

Since, for example, I don't know or trust ppa:chris-lea/redis, I'm leery of installing his PPA on my machine for following the latest Redis Server versions. If Redis created its own PGP-signed PPA for Ubuntu and Debian, this would bypass the problem and assert users of secure, signed Redis packages directly from the makers of the software.

Comment From: naftulikay

Problem can easily appear in places like this which mindlessly pulls from ppa:chris-lea/redis and yet is shown as a "trusted build" for Docker. Having a Redis team-owned and PGP-signed repository, Docker containers like this would be fairly trivial to update/make work.

Comment From: pilwon

:+1: I agree there should be an official PPA for redis.

@rfkrocktk dockerfile/redis no longer uses ppa:chris-lea/redis. It now builds from the latest stable source code.

Comment From: pilwon

@antirez There's a tool such as FPM that makes package creation a breeze.

Comment From: naftulikay

:+1: for FPM, great tool. Just sign them using a highly-secret internal private Redis team GPG key. The kind that you only key on a hardware smartcard.

Comment From: kutchar

+1 on having an official repo

Comment From: makmanalp

+1 on this.

Comment From: kutchar

An option would be to see if @chrislea would be willing to help create an "official" redis repo.

Comment From: chrislea

For the record, I'm totally happy to help if I can on this, assuming we'd be sane about what distributions would be targeted for support. Specifically, if we support what pbuilder supports, no problem (that would cover all current supported Debian and Ubuntu releases).

As an additional note, such a repo would I think necessarily include jemalloc, so there might be some considerations there, though I'm already doing this with my PPA so as a litmus test this seems to be safe.

Comment From: chrislea

As an additional, additional note: the bit about including jemalloc is only needed if the goal is to conform to the Debian zen of how to package things (don't bundle or statically link libraries is a big deal in that zen).

If that wasn't on the table, then it's actually easier to use the included jemalloc that redis ships with for the builds, compared to what I'm currently doing.

This is non-trivially what I'm doing with my involvement with Node.JS / @nodesource packaging, as we maintain our own Node repositories there where we statically link in lots of things (V8, cares, etc) that don't adhere to the Debian packaging guidelines.

Comment From: kutchar

Thanks for the prompt response @chrislea. I think the latest versions of Ubuntu/Debian should cover a good chunk of use cases out there, it's definitely much better than not having an official repo at all. As far as jemalloc, I don't think it'd be such a big deal to just compile it in statically, specially since it will be more consistent with the "non pacakged" redis.

I'm assuming the next step would be to see what @antirez thinks.

Comment From: naftulikay

I'm excited to see what happens next. Any updates?

On Mon, Sep 29, 2014 at 9:35 PM, Drew Kutcharian notifications@github.com wrote:

Thanks for the prompt response @chrislea https://github.com/chrislea. I think the latest versions of Ubuntu/Debian should cover a good chunk of use cases out there, it's definitely much better than not having an official repo at all. As far as jemalloc, I don't think it'd be such a big deal to just compile it in statically, specially since it will be more consistent with the "non pacakged" redis.

I'm assuming the next step would be to see what @antirez https://github.com/antirez thinks.

— Reply to this email directly or view it on GitHub https://github.com/antirez/redis/issues/1732#issuecomment-57266106.

Comment From: listepo

+1

Comment From: halida

+1

Comment From: charlesmims

+1

Comment From: prozacgod

+9001

Comment From: michaeljoy

+∞

Comment From: antirez

I'm not against this proposal, but we currently don't have bandwidth to handle that. If there are at least two Redis users that may provide this service, would be cool. Otherwise we need a way in order to make this automatic. I've no idea about how to do this since I'm not a packages expert, however at download.redis.io we have the latest release available, and there are SHA1 signatures at github.com/antirez/redis-hashes. So if you provide me with a set of commands to run in order to automatically generate the package from the tar.gz so that this is fully automated at every new release, I can do it.

Comment From: naftulikay

@chrislea Would you be able to help @antirez (and indeed, all of us) by showing them how to automate the packaging into a PPA based on what you've done in your own repository?

Comment From: chrislea

Well there are still some manual steps involved when I make a new package. Minimally, you have to make a new changelog entry, and you also have to check that the patch to make redis use the system jemalloc instead of the bundled one still works, and update it if it doesn't. So I don't know if it can be fully automated the way things currently are. It's not a lot of work, but it does require human eyes on it.

On Fri, Dec 19, 2014 at 10:39 AM, rfkrocktk notifications@github.com wrote:

@chrillea Would you be able to help @antirez https://github.com/antirez (and indeed, all of us) by showing them how to automate the packaging into a PPA based on what you've done in your own repository?

— Reply to this email directly or view it on GitHub https://github.com/antirez/redis/issues/1732#issuecomment-67678215.

__ https://chrislea.com http://about.me/chrislea 310-709-4021

Comment From: chrislea

I've been thinking about this a little more over the last few days and wondering if there's a way to make this trivial for @antirez. If we use the bundled jemalloc then it should in theory be possible. But there are still some questions on the table. - Where will the packages be hosted? Is it a PPA on Launchpad or is it self hosted? - If it's a PPA, are we sure we're okay supporting Ubuntu and not Debian? - If it's self hosted, what distros and architectures are to be supported? Can we support ARM? Should we? - If it's self hosted, do we need to provide some sort of setup script to help people add that repository to their sources lists since add-apt-respository won't always be available? - Should any such installation script additionally try to make things work with Ubuntu / Debian derivatives (Linux Mint, etc)?

These are all things we had to make decisions about with making NodeJS packages so I'm just bringing them up here as well.

I should also point out playing devil's advocate that my current PPA is a signed repository as required by Launchpad. If anybody is really concerned I'm tampering with the sources, writing some script to download the .orig.tar.gz file that gets uploaded for each release and comparing the SHA1 sigs of all the files to the officially posted ones should I believe provide a full audit trail (though if you want to go full neck-beard you'd need to do the same with the jemalloc sources and then inspect the patches I / Debian use when building the packages). My point being, really, that if you trust Debian / Canonical to not tamper with the sources, it's probably safe to trust me as well. In both cases the repositories are signed and if I was doing anything bad, it would be trivially easy to trace it back to me for any given release.

Comment From: michaeljoy

I think the broader concern is just one of trust and repeatability. Source builds are the ultimate in 'trust', but not the source of pure repeatability for a variety of system environment reasons.

If there's a build system like Jenkins et. al. available it's just a matter of adding another build step that gets kicked off after the build makes the final artifact .tar.gz et. al.

As for the jemalloc debate, it honestly should be included as to ensure the server is running the same version the redis package is built and tested against on an ongoing basis. System packages are honestly rarely kept up to date (see ruby, perl, java, go, python and other dependencies in debian land) and not appropriate for stable and secure services as they don't get patched often, or at all by the upstream providers.

The repo issue is actually pretty simple as that can be handled as part of the build (gradle has a pretty slick debian package plugin) as well without having to maintain an active state of the repo (it just downloads the indexes, increments the new binary, and updates the indexes for each package). For self-hosted, AWS S3 works brilliantly, although there's a variety of other means as well if more traditional web servers are available. The deb-s3 gem works quite well for signing and uploading the package.

If someone is concerned about making it 'easy' to add the repo, puppetlabs has a repo package that's a prime example of how it 'can' be done, but not necessarily should. A simple curl of the gpg key from the appropriate hkp enabled key mirror and echo of the repo into sources.list.d is typically enough to get nearly anyone going.

But yes, this should be fully automated. Down the line, a package install test, can be also devised and automated to ensure the finished product is viable with no need for human intervention unless it's busted. Human's aren't meant to push buttons repetitively, I think that's why we get injured when we try :)

Really when it comes down to it, the tooling used for automating the packaging and uploading has more to do with what the maintainer of said process is most comfortable with (there's no 'best' tool really).

My 2 cents. Let me know how I can help.

Comment From: naftulikay

@michaeljoy Exactly, trust and repeatability. People would install a repository for the following reasons: - They don't want to have to build the software. - They want to get a compiled binary provided by the software developers with a reasonable level of trust. - When security problems are inevitably fixed, it shouldn't mean SSHing into all of your Redis machines, downloading source, recompiling, etc. You should be able to just apt-get update && apt-get install redis, and you could do this pretty easily with mcollective.

As far as providers, like @michaeljoy it doesn't really matter where the repository is hosted, so long as it's signed and it works.

In response to @chrislea and his questions: - Where it will be hosted shouldn't matter too much. Launchpad is easy enough, right? - PPAs don't support Debian? Can't you specify that you're building the package for "trusty" and "wheezy" at the same time? Without knowing too much of the internals of add-apt-repository, I'm thinking it just downloads and installs the PGP key for the repository, installs the source reference, and done. If PPA can't support Debian, then self-hosted is the way to go. repo.redis.io? - For the distributions to support, first start with the current Ubuntu LTS releases 12.04 and 14.04 in addition to Debian's stable(s). AFAIK we can support ARM, and it'd be a "nice-to-have" in case someone's building a server farm out of Raspberry Pi's. Barring any significant differences, supporting latest Ubuntu derivatives should work reasonably well even on desktop. - If it's self-hosted, a setup script is just another "nice-to-have." I'm sure that most people are just interested in downloading the PGP signing key (over HTTPs, for obvious reasons) and installing it, then adding the repository source files. Eventually providing a setup script (echo "rm -fr /" | sh -) would be good for most users.

Also @chrislea I apologize if my not trusting you and your packages was offensive in any way, I just don't know you. I trust Debian and Canonical to have reasonable security measures in place it is therefore reasonable to trust them. If they were compromised, everyone would be owned, because as a user of their operating system, you more or less must trust them.

Picture the following scenario: you're responding to an intrusion on one of your servers at your company. Long story short, you installed some PPA and the PPA was compromised because the user didn't have a password that wasn't password. Imagine explaining to your security department that you installed some PPA from some guy that you never met because "it was easier" than compiling from source. If the compromise came from Debian or Canonical, this would be significant and probably wouldn't end up with you losing your job. The same goes for Redis: if you installed the official Redis PPA/repository which is provided directly from the developers and that was compromised, you'd still probably not lose your job because what you did was sane and reasonable.

So, @chrislea it's not that I don't trust you, I just want to keep my job ;)

Comment From: kutchar

A relatively painless and pragmatic (or stop-gap) approach here could be to ask @chrislea to just create a new PPA that only hosts redis and take on the responsibility of publishing the redis packages there. (Sorry @chrislea if I'm asking too much, but seems like you're already doing the work, I'm just trying to formalize it and get you recognition). Then, for the trust part, all we would need would be for @antirez to trust @chrislea and declare this PPA as the official repo for redis on redis.io (kind of like Linus and his lieutenants). Considering time is a valuable commodity, this to me seems like quite a lot of value with not much pain. Any thoughts?

Comment From: chrislea

If that works for people then it's already done. My existing redis PPA only has redis and jemalloc in it. I generally keep all my PPAs separate since I package tons of stuff.

On Tuesday, December 23, 2014, Drew Kutcharian notifications@github.com wrote:

A relatively painless and pragmatic (or stop-gap) approach here could be to ask @chrislea https://github.com/chrislea to just create a new PPA that only hosts redis and take on the responsibility of publishing the redis packages there. (Sorry @chrislea https://github.com/chrislea if I'm asking too much, but seems like you're already doing the work, I'm just trying to formalize it and get you recognition). Then, for the trust part, all we would need would be for @antirez https://github.com/antirez to trust @chrislea https://github.com/chrislea and declare this PPA as the official repo for redis on redis.io (kind of like Linus and his lieutenants). Considering time is a valuable commodity, this to me seems like quite a lot of value with not much pain. Any thoughts?

— Reply to this email directly or view it on GitHub https://github.com/antirez/redis/issues/1732#issuecomment-68010636.

__ Sent from phone, please excuse brevity...

Comment From: naftulikay

Create a new PPA called ppa:redis/stable and I'm down. It can be the same packaging you're doing at ppa:chrislea/redis.

Comment From: chrislea

Ah, okay, I think I misunderstood a little bit but get it now. So there are basically two scenarios that might work here. 1. @antirez designates me as a trusted person to make the packages and I then make them in some official capacity. I'm fine with that if he is. 2. I write up a condensed version of the Debian packaging guidelines that explains how I make my packages and @antirez (or somebody) scripts those steps up so he can do it easily.

For case 1: I can't make a PPA called ppa:redis/stable because the way that Launchpad names things is ppa:<Launchpad username>/<PPA name>. So in that case, @antirez would need to set up the official account on Launchpad and then give me access to the GPG key so I can sign everything. There is already a redis user on Launchpad, I don't know if it's him or not. But the point is, the account should be under his control so that it is known to be appropriately blessed.

For case 2: I'd just need to find a few free hours to write something cohesive up. I could probably get it done pretty soon as it's the holidays for me right now and things are relatively slow, but no promises.

There are two more things that people may or may not care about I think. First, these options mean no builds for Debian proper. Launchpad will only build for Ubuntu. Second, Launchpad will only accept builds for currently supported Ubuntu distributions. As soon as one slips outside the support window, Launchpad will reject new builds for it.

I'm good with either of the two cases, just let me know what works for people.

Happy Holidays all!

Comment From: naftulikay

@antirez Can you collaborate with @chrislea to make this a reality? I'm happy with either solution, but I'd prefer case number 2. We could potentially even write a little script called "upload-redis-release" which could create a signed DEB for distribution on Launchpad.

As far as Debian compatibility, it stinks that we have to lose it for the time being for the ease-of-use of Launchpad, but IMO it's better to get something online which suits the vast majority of users. I could be wrong and I don't have statistics in front of me, but I think it's a fair bet that of Debian-based server OS's out there, I would think that the majority would be Ubuntu.

A later effort should be made to setup a dedicated repository to expand support to other distributions and Debian proper. When there's a big enough demand for it, we could use the packaging scripts provided by @chrislea to make it easy to ship releases to a Debian repository owned exclusively by Redis.

Comment From: xeoncross

I'm about to build a bash script to automate this build if no more has been done on the PPA/DEB front.

Comment From: naftulikay

I haven't heard anything back, so feel free to get started.

Comment From: chrislea

Ditto, haven't heard anything. I'm happy to help however I can.

Comment From: kutchar

I'm confused. @chrislea don't you already have the scripts that @Xeoncross wants to build?

Comment From: chrislea

Well, I'm not sure exactly what @Xeoncross is going to whip up, but I don't have any scripts for building this. I have a debian directory that has the files you need to actually make a .deb package for Redis, but I haven't written any custom scripts for this. That said, the debian directory in question is available to anybody from the existing PPA on Launchpad.

Comment From: xeoncross

I usually write bash scripts for modern packages that I need. I was going to just copy the quickstart... then I realized that I should just use your PPA @chrislea since my use-case is a non-private database and you participating in this discussion gives me more trust in your PPA.

Plus, I keep trying to move to ansible so I'll stop writing bash scripts.

Comment From: yvbeek

+1 ppa:redis/stable at launchpad sounds like a great idea, @antirez what are your thoughts?

Comment From: simshi

+1 for official stable ppa

Comment From: zerthimon

+1

Comment From: adik

+1

Comment From: DaAwesomeP

+1

Comment From: FabianKoestring

+1

Comment From: halida

+1

Comment From: madphenomenon

+1

Comment From: kutchar

https://bintray.com/ or https://packagecloud.io/ could be viable options too instead of a PPA

Comment From: abcfy2

+1. Need official linux repo.

Comment From: lafengnan

+1

Comment From: llbbl

It has been +16 months since original ticket and looks like 7 months since any real effort/updates. Can we all please have a status update at least? Thanks.

Comment From: llbbl

Hmm.. I found redis-server on ubuntu. http://packages.ubuntu.com/trusty/misc/redis-server

Comment From: kutchar

@llbbl redis-server has been available on Ubuntu for a while but the version is old. The reason for this issue is to have an official redis repository to publish the latest stable releases.

At this point, we pretty much have all the items ready to go forward with an official repo except @antirez 's approval.

Comment From: sirkubax

+1

Comment From: zerthimon

So, currently, it seams this issue is blocked by lack of communication between @chrislea (who is up to help) and @antirez (who doesn't seem to care)

Comment From: DaAwesomeP

You know, even a package that simply downloads and builds Redis every time it is updated wouldn't be too too bad considering how quick it is to build Redis. There would have to be a redis-builder command line tool that could be set to automatically build and upgrade Redis or only when prompted too. It's not as practical as a real package, but if this isn't going anywhere, then it might be an option.

Comment From: ariscn

@antirez You have volunteers willing to help you automate the PPA process - could you please reply describing exactly what you need from the community in order to make this happen?

Comment From: chrislea

So for everybody on this issue: let's please not start yapping at @antirez over this. I'm sure that making packages that are easy for you to install a) isn't in his job description and b) is a very low priority thing for him what with all of the "making redis work as stupendously well as it does" time that he puts in.

All of this started because people wanted an "official" repository for redis. So I'd say for the people who care about this, maybe it's a better use of energy to think about what you'd need to see to trust a repository as official. Keep in mind, you're probably already doing this since your Linux distro of choice ships redis to you and you trust it, since you trust your distro provider.

If there's consensus about what that repo would be, I'm happy to publish packages there in addition to (or possibly instead of) my PPA.

$0.02.

Comment From: DaAwesomeP

You could make a redis user on the OpenSUSE Build Service. You can use OBS to automatically compile and host packages for Debian, Ubuntu, OpenSUSE, Fedora, CentOS, Arch, and I think a few more. It hosts the proper format for every OS and package manager type (apt for Debian/Ubuntu, yum for Arch/Fedora/CentOS, zypper for OpenSUSE). It's not PPA, but it will accomplish the same goal of having a place to host up-to-date packages under an organization name of redis.

All of this started because people wanted an "official" repository for redis. So I'd say for the people who care about this, maybe it's a better use of energy to think about what you'd need to see to trust a repository as official. Keep in mind, you're probably already doing this since your Linux distro of choice ships redis to you and you trust it, since you trust your distro provider.

If you consider "official" as a repository maintained by the official people and hosted by a very trusted source (cough_GitHub_cough), then OBS will work. OBS operates easily with a spec file. Once it's set up, updating a package is as simple as uploading a new release archive. It will then automatically compile, package, and publish it (assuming that it built correctly).

There are other services like OBS, but this is just the one that I've used before and really like.

Comment From: orykami

+1

Comment From: omarkhd

+1

Comment From: schaary

+1

Comment From: sandstrom

+1

I know that Vagrant and a few others use Bintray, which seems to work really well. Free for open-source projects. I'm not affiliated with them, just thinking it may be worth looking into since it seems simple enough.

Comment From: zerthimon

So I'd say for the people who care about this, maybe it's a better use of energy to think about what you'd need to see to trust a repository as official. @chrislea: Ideally, some similar agreement like this one with nodesource ( https://nodesource.com/blog/chris-lea-joins-forces-with-nodesource ) would be best. But, I think that just by having @antirez acknowledging (officially? perhaps mentioning it on redis.io/download with a link to your ppa ?) that he trusts you as a maintainer of the official redis ubuntu packages and your ppa as an official repository for official redis ubuntu packages, would be enough for most participants in this thread.

Personally I'm already using your ppa for all my redis installations.

Comment From: chrislea

Yup @zerthimon +1 on that, basically been my Zen for a while now. Making these packages is, at least for now, really pretty easy so I plan to continue doing it for the foreseeable future.

Comment From: windhamdavid

@chrislea Thank you for the :package:

Comment From: rpardini

First thanks @chrislea for the the ppa. A quick question, it seems launchpad only keeps the latest stable version available (right now it's 3.0.6 I think). I'd love to be able to install eg 2.8.24 (latest "old" version). Do you envision a way to reproduce the older builds somehow? I can surely re-package those... Thanks.

Comment From: chrislea

Unfortunately that's a Launchpad thing. When you push a new version in, it takes out any previous versions so long as the builds succeed. To keep the 2.8.x line going would require a separate PPA just for that series.

Comment From: antirez

Hello, if there is agreement about what the main PPA linked from redis.io should be, I've no problems doing it. I want Redis to remain distribution agnostic, and also to avoid having another overhead in the project, so I don't think that I can help handling officially anything like that, but if it's something many people need, to provide a link to the best "community PPA" is probably the way to go.

Comment From: bhat3

FYI getting packages trusty (14.04) is just a no source changes backport for xenial (16.04). Just did it with http://archive.ubuntu.com/ubuntu/pool/universe/r/redis/redis_3.0.6-1.dsc and it worked flawless (including the nice testsuite).

@chrislea Want about bringing it to trusty-backports? Would join force but you're the maintainer :)

Comment From: prascuna

Hi everyone, since no official debian/ubuntu repo is available I found out that building the package by myself was the best option. To simplify the process, I created a Docker image that in one step allows to download redis source code and build it from scratch. Hope it helps:

https://hub.docker.com/r/prascuna/docker-redis-packager/

Comment From: bhat3

FYI i will find the backported package from xenial to trusty over here https://launchpad.net/~trio-interactive/+archive/ubuntu/unstable/+sourcepub/6477748/+listing-archive-extra

Altough i can't recommend to use this PPA, but you can grab the no source changes backported package :)

Comment From: prascuna

@bhat3 it's not the latest version unfortunately

Comment From: bhat3

@prascuna You mean not the latest upstream version, but it is still the newest Ubuntu package backported -> http://packages.ubuntu.com/source/yakkety/redis . I took it from xenial cause you can only do LTS to LTS backporting in the long support run. Backporting from a regular release would mean that you won't have updates to backport once the release is EOL while the LTS still is supported. That's why we stick to xenial's package as we run it on trusty in production.

Comment From: bhat3

@prascuna Oops i overlooked the new versions Chris has in Debian. For that mistake i justed uploaded them to our unstable PPA but without any testing or guarantee of following updates: Xenial: https://launchpad.net/~trio-interactive/+archive/ubuntu/unstable/+sourcepub/6480753/+listing-archive-extra Trusty: https://launchpad.net/~trio-interactive/+archive/ubuntu/unstable/+sourcepub/6480751/+listing-archive-extra

Comment From: shaharmor

@chrislea any reason why your PPA isn't updated with the recent versions?

Comment From: chrislea

There's two principal reasons. 1. The new versions of Redis bundle a newer version of jemalloc than what ships in modern distros, and I haven't had time to investigate what the best way to deal with that is. 2. There's a test that consistently fails on all the builds I've tried, and I haven't had time to look into that properly yet.

On Sun, Aug 14, 2016 at 7:11 AM, Shahar Mor notifications@github.com wrote:

@chrislea https://github.com/chrislea any reason why your PPA isn't updated with the recent versions?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/antirez/redis/issues/1732#issuecomment-239675586, or mute the thread https://github.com/notifications/unsubscribe-auth/AAHD_vO5Ly2WMl7H30-WSZ2lGFhzQEvdks5qfyIRgaJpZM4B5VcB .

__ https://chrislea.com http://about.me/chrislea 310-709-4021

Comment From: bhat3

@chrislea @shaharmor What versions are you speaking are you speaking about?

Comment From: shaharmor

@bhat3 at https://launchpad.net/~chris-lea/+archive/ubuntu/redis-server the latest version is 3.0.7, while redis is already at 3.2.3

Comment From: bhat3

@shaharmor Ah okay i tought you're speaking about a newer version than in Debian sid. @chrislea At least with 3.2.0-3 i didn't run into issues with jemalloc on trusty and xenial, but i need to retest with 3.2.3-1.

Comment From: chrislea

I think I may not have been clear.

In the new 3.2.x versions of redis, the version of jemalloc that is bundled is 4.0.3. The version of jemalloc that ships with say Ubuntu Xenial is 3.6.0.

I have always used the system jemalloc instead of the bundled one with these packages because that is what the Debian packaging standards expect you to do. That may no longer be the preferable choice, but I haven't had time to look into it. So that's one reason why I haven't bumped the version in the PPA to the 3.2.x series.

The other is that failing test I mentioned.

Comment From: bhat3

@chrislea Now i got you :) And yes that's the typical backporting dependency hell i did run into for years with serveral PPAs ;) I think for the PPA itself it would be technical the best solutions to ship a bundle, as we then don't mess around with the system jemalloc when users just want a new Redis and only for that a newer jemalloc. But for Debian and security reasons that's quite probably not the way to go.

That's the problem i see, that we got different approaches to packaging whether it's for the next distro release or for backporting to old distro releases. I did just remember some packaging for the lucid release where i created new package for trusty and tried to build the bridge by not going with the latest and cool Debhelper packaging optimization for being able to still backport the package easily. For maintenance reasons i for example like to push a newer package version for an upcoming distro release and then do a "no source changes" backport, but that's in many cases not really possible and is also the reason why Flatpak/Snap are getting popular ;)

For me the main question is now also does Redis really depend on jemalloc 4.0.3? I just did a really quick look a the buildlogs of the 3.2.0-3 backports and even on trusty it seems to be even fine with a 3.5.1 version of jemalloc.

Comment From: martinhoefling

Debian sid builds redis 3.2.3 vs jemalloc 3.6.0. I guess @chrislea is referring to

   * [NEW] Jemalloc updated to 4.0.3 (Salvatore Sanfilippo)

in the redis releasenotes.

Comment From: bhat3

@chrislea @martinhoefling Have you spoken with upstream about this already? Unfortunately i am sick ATM and can't push it right now, although i need to consolidate our Redis installations and backports soon :)

Comment From: Apollon77

+1 for a current version in a (trusted) ppa -like chris lea :-) ) or a stable repo ... DO we have any news?

I currently use 3.0.6 on my ARMHF Cubietruck and experience a memory problem ... so first try would be a more current version :-)

Comment From: Apollon77

Were anyone able to jeck the real jemalloc dependency?

Comment From: bhat3

Didn't had a look so far but after next week i can probably spend some time on Redis packages again for one of our customers :) But no promise so far, but i will post when i start packaging.

Comment From: Izopi4a

+1

Comment From: bhat3

@Izopi4a Thanks for the reminder ;) Am finally in vacations and we needed to postpone the project quite a bit ... but it's still on the ToDo list and i keep you updated once there's progress :)

@all But what ever i or someone else does with the packaging and backporting having an at least halfway maintained PPA is still not reached by that. Would vote for a teambased PPA where a few people can takeover with lazy pushing it. Who would be in as well?

Comment From: poige

I'm strongly against to have any package with jemalloc built-in. Jemalloc is pretty under active development, it's much more easier to use its up-to-date version with ld.so.preload if needed, instead of abandoning use of packaged redis version just due to it's shipped with some ancient version that also may cause troubles being used with new one added with ld.so.preload. No way, it's stupid.

Comment From: bhat3

@poige Valid point, but as you said under active development also means that there might be some breakage or does jemalloc upstream stick to the big QA effort of keeping a stable ABI/API? Did you really get the problems with packaging new Redis version that need a newer jemalloc than Debian/Ubuntu have, if yes please share your knowledge :)

Comment From: poige

Take a look at the project, it's on github — they look solid: unit tests and moreover, — it's memory allocator, — the thing that's being heavily used by lots of different software in the system so bugs are honey (s)potted.

But even if we'd suppose that they may ship bugs in their releases (anybody can), it's system maintainers' responsibility which version of Jemalloc they're gonna use, when-why and so on: they can choose old version, but easily can switch to recent stable. This flexibility is absent for built-in approach.

Comment From: bhat3

@poige Sorry but nobody questions the quality of jemalloc, quite the opposite. So please don't do such arguments. The problem is here that Redis ships with a newer version of jemalloc that's AFAIK still not in Debian and here you first argument goes along with the Debian policy that you must use commonly shared libaries from Debian, but they don't meet the version requirement of current Redis. If i would just find more time for that issue :(

Comment From: poige

You're misreading my answer obviously. Specially for you just once more:

But even if we'd suppose that they may ship bugs in their releases (anybody can), it's system maintainers' responsibility which version of Jemalloc they're gonna use, when-why and so on: they can choose old version, but easily can switch to recent stable. This flexibility is absent for built-in approach.

Comment From: bhat3

@poige Missreading and not following an argument are two things. But please let's spend the energy and time on providing a solution for Redis and Debian and stop this missleading discussion. Redis got a normal version dependency no matter if they ship jemalloc or not. With Debian packages this version dependency can't be fullfilled. That's the classical backporting dependency hell that flatpak and snap try to solve.

Comment From: poige

@bhat3 you're simple messing things up and not following any logic. I'm talking about (and against) build-in jemalloc into executable of Redis.

Comment From: chrislea

Okay, after toying around with it all for a while, I decided to go ahead and build the current version of redis (3.2.6 as of typing this) with jemalloc statically linked in. This breaks from my previous decision to use the system provided library. It should not have any bad effect on any end users though as this change does not require them to install any new packages or anything.

Probably the key deciding factor for me was this blog post from @antirez from back in the day. In there, he notes that they are shipping jemalloc with redis, which was not a decision that they made lightly. I figure if that's the way he recommends people use his software, and given there are some complexities involved with doing it the previous way, it's time to make the switch.

Waiting on Launchpad to build things now.

Comment From: poige

Stupidity wins another time. Quick question for you: jemalloc has lots of compile-time knobs. One of them is off by default and it's called "enable-munmap". It's disabled since according to jemalloc's author "Linux VM has quirks" in regards. OTOH, it's known that w/o munmap some software tends to bloat its VSS beyond any sane bounds. So you're jumping in as a green newbie and what you decide?

Comment From: chrislea

Wow @poige , okay, let's see...

1. I've been building redis (and other) packages for people on Launchpad to use since around 2008, so I think I've probably grown past the "green newbie" phase.

2. I am building redis in the way that the author of redis suggests the software be built on Linux.

3. The config section for jemalloc as it was built for these packages looks like this:

jemalloc version   : 4.0.3-0-ge9192eacf8935e29fc62fddc2701f7942b1cc02c
library revision   : 2

CONFIG             : --with-lg-quantum=3 --with-jemalloc-prefix=je_ --enable-cc-silence 'CFLAGS=-std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops ' LDFLAGS=
CC                 : gcc
CFLAGS             : -std=gnu99 -Wall -pipe -g3 -O3 -funroll-loops  -fvisibility=hidden
CPPFLAGS           :  -D_GNU_SOURCE -D_REENTRANT
LDFLAGS            :
EXTRA_LDFLAGS      :
LIBS               :  -lpthread
TESTLIBS           :
RPATH_EXTRA        :

XSLTPROC           : false
XSLROOT            :

PREFIX             : /usr/local
BINDIR             : /usr/local/bin
DATADIR            : /usr/local/share
INCLUDEDIR         : /usr/local/include
LIBDIR             : /usr/local/lib
MANDIR             : /usr/local/share/man

srcroot            :
abs_srcroot        : /home/chl/Packages/Redis/foo/redis-3.2.6/deps/jemalloc/
objroot            :
abs_objroot        : /home/chl/Packages/Redis/foo/redis-3.2.6/deps/jemalloc/

JEMALLOC_PREFIX    : je_
JEMALLOC_PRIVATE_NAMESPACE
                   : je_
install_suffix     :
autogen            : 0
cc-silence         : 1
debug              : 0
code-coverage      : 0
stats              : 1
prof               : 0
prof-libunwind     : 0
prof-libgcc        : 0
prof-gcc           : 0
tcache             : 1
fill               : 1
utrace             : 0
valgrind           : 0
xmalloc            : 0
munmap             : 0
lazy_lock          : 0
tls                : 1
cache-oblivious    : 1

1. This is the same way that the standalone jemalloc package that ships with Debian / Ubuntu is built, so it's the same as if the package was still dynamically linked.

2. If you don't like the way I'm building these, feel free to not use them.

Comment From: poige

1. great. Surely, I didn't use your builds before and unlikely would be using them since as I said before it's stupid to build in something that's way too tuneable and at the same time load dependent. Currently I'm seeing MySQL's with VSS around 170 GB (and RES around 20 GB) that I'm sure was due not using munmap (although I can be mistaken). I also can imagine it would work just nice for others who have different load pattern. The great thing it was LD_PRELOADed so it can be relatively easy fixed — at least I can easily go with a try to fix it

2. https://github.com/sysown/proxysql/issues/612

   This setting, default on Linux, seems to cause a lot of virtual memory allocated under certain workload.

3. There're lots of others mem. allocators. For e. g.: https://www.reddit.com/r/programming/comments/18zija/github_got_30_better_performance_using_tcmalloc/

building in one of them effectively prohibits people from using others. To be more precise they couldn't even know that they're using a mix of them having all kinds of quirks and unstable results due to this package maintainer's choice.

1. Probably having 2 version of packages with different names like redis and redis-jemalloc would be the best possible way to handle this. But choosing between them (if I'd have to) I'd prefer more flexibility taking into consideration that users would be able to use any allocator they need using LD_PRELOAD

Comment From: bhat3

@poige Please dude, this about a Debian package and yes it's not nice to do statically linking, but get over it and use Gentoo, LFS or maintain your own custom builds. Calling "stupidity wins again" when Chris finally found some time for doing this and you just talked isn't nice! @chrislea But you won't get them into Debian like this or?

Comment From: poige

When stupidity wins it's not nice either. Dude^W @bhat3

Comment From: Apollon77

@chrislea Great work man! Waited for it, Any chance to also get packages for armhf architecture?!

Comment From: PelleRavn

Awesome work @chrislea, thanks a million! 👍

Comment From: sjuxax

@poige If jemalloc needs a compile-time tuning flag that depends on the application's workload, isn't that a stronger case for statically linking it? Some applications will not suffer when jemalloc uses the system to manage memory and some applications will, so shouldn't each application distribute a library that's tuned to serve its workload? That makes sense, right?

The --disable-munmap flag instructs jemalloc to keep track of memory on its own instead of releasing it back to the OS, which can have its own serious implications. It appears that like most other applications that hoard memory and manage it internally, the motive is to reduce memory fragmentation, not memory usage (VSZ). Whether that's useful is going to be application-dependent. For example, it may be a good idea for database servers, but it may not be a good idea for web browsers like Firefox (which also uses jemalloc).

I understand that in principle, most Linux distributions dynamically link everything. However, this is a third-party PPA and the goal is to get working, recent builds to users who choose to use it. If you dislike the options used in the build, you can build yourself or use the (badly-maintained) packages in the official repository. There's no need for hostility.

@chrislea Keep it up! I don't know what Ubuntu's deal is, but they seem to have a very hard time keeping their redis package either sane or secure. Your PPA is great.

Comment From: chrislea

To be clear @sjuxax , there is no "deal" with Ubuntu keeping Redis up-to-date. Most of the major distro vendors (Ubuntu, Debian, RedHat, Fedora, etc) have a policy that they almost never bump versions of things for a given release. The idea is that if you get your software working on say Ubuntu Xenial, then nothing is ever going to change underneath you that might break things. If you want the distro you're using to always have the newest versions of software I'd recommend using a rolling release distro like Arch Linux.

All that said, I'm glad you're finding my Redis PPA useful. :)

Comment From: sjuxax

@chrislea Yeah, I've been a user of all three, Debian, Ubuntu, and Arch Linux, for over a decade. I'm aware of their general packaging philosophies (which, despite insistence to contrary, are not really that consistent from any side).

Nevertheless, Ubuntu takes pretty bad care of Redis. xenial is using upstream's 3.0.6 despite 3.0.7's release 3 months prior to 16.04 "going gold". It is conventional to release minor bugfix releases, that's why there's usually stuff to update when you apt-get update and apt-get upgrade; it's not purely security-related. trusty hosts 2.8.4 with several unpatched security vulnerabilities that are being actively exploited in the wild; the security patches have not been backported to the Ubuntu package. Debian seems to do a much better job tracking upstream redis for bugfix releases and issuing security-related backports when appropriate.

Comment From: poige

@sjuxax your note about "no need for hostility" is somewhat stupid (again). There's no hostility in considering things stupid or silly or lacking sense and so on — in other words, speaking freely isn't hostility by itself (although modern western state of thinking as it seems is all about not speaking freely what's on mind and then spending bucks for shrinks to receive absolution). I gave up this thread quite a long ago since people tend to stick to theirs decisions even if they're not optimal. "Works for me and why bothering and let's compile another cool stuff in just due to it's cool and/or everybody does that" that's the motto. So in fact I didn't care until you came and called me.

As to your input on jemalloc's compile flag -- I wrote previously that correct way to handle this is having 2 versions of packages: redis-jemalloc and just redis. Most of IT crowd would agree that principle of least surprise is basic. When you get not only redis but also jemalloc w/o having any clue it was compiled in, well, it's violation of the principle (at least).

Comment From: bhat3

@sjuxax Absolutely right about Ubuntu, but that's a general problem of [universe] and that's why it's so common to use PPAs. I just cherry pick from Chris for xenial and trusty and pushing it to our servers via an own PPA. If you don't mind getting a bit dirty with packaging some times it's a decent approach to overcome the release cycles and still using debian/rules for software distribution ;) @chrislea To repeat myself a bit, do you think there is any chance to push your backports with static jemalloc to the trusty-backports and xenial-backports repos? That we would be at least the "offical" Ubuntu repo style for handling this. It does quite collide with https://www.debian.org/doc/debian-policy/ch-sharedlibs.html but as discussed i go along. Although the nicest solution would be a co-installable versions of jemalloc by package naming like we got it with PHP thanks to Ondřej.

Comment From: chrislea

@bhat3 I have zero influence / control / options when it comes to the official *-backports repositories. They are controlled by Canonical, so I can't put things in there. Also, I'm pretty sure these packages would get rejected on principal because of the shared libs thing.

Comment From: sandstrom

@chrislea Thanks for these packages! 🏅

Comment From: varnav

3 years since issue is opened it is sad to see that there is still no official Ubuntu repo for Redis.

Comment From: suslikas

I'm spend some time to found how to install latest Redis version to 16.04, so I'm very surprised, is it problem to find trusted repository... May be someone will use my solution for next 3 years...

# mkdir redis
# cd redis
# wget http://ftp.lt.debian.org/debian/pool/main/r/redis/redis-sentinel_4.0.1-2_amd64.deb
# wget http://ftp.lt.debian.org/debian/pool/main/r/redis/redis-server_4.0.1-2_amd64.deb
# wget http://ftp.lt.debian.org/debian/pool/main/r/redis/redis-tools_4.0.1-2_amd64.deb
# apt-get install libatomic1 libjemalloc1
# dpkg -i *.deb
# redis-server -v
Redis server v=4.0.1 sha=00000000:0 malloc=jemalloc-3.6.0 bits=64 build=467e3c6dafb2a43

Comment From: zerthimon

Just use Docker and official image at https://hub.docker.com/_/redis/ Package managers are old :))

Comment From: dalvarezquiroga

+1 , official PPA :-)

Comment From: shravan2x

@antirez 2018 is here; can we make progress on this?

Comment From: geraldhiller

+42 for an official PPA :-)

Comment From: obsergiu

+43 for an official PPA 👍

Comment From: orlra

+44 for an official PPA +1

Comment From: markusr

+45 for an official PPA +1

Comment From: suhienet

+46 for official PPA

Comment From: suhienet

+46 for official PPA

Comment From: sandstrom

With Debian Stretch there is now a package: - https://packages.debian.org/stretch/redis-server

Same goes for Ubuntu Bionic 18.04: - https://packages.ubuntu.com/bionic/redis

Comment From: shravan2x

@chrislea This is probably not the place to report this, but your PPA for Ubuntu 16.04 Xenial is currently broken. The systemd service file expects some systemd functionality that the default systemd on 16.04 doesn't support.

Nov 23 21:31:49 ubuntu systemd[1]: [/lib/systemd/system/redis-server.service:30] Unknown lvalue 'MemoryDenyWriteExecute' in section 'Service'
Nov 23 21:31:49 ubuntu systemd[1]: [/lib/systemd/system/redis-server.service:31] Unknown lvalue 'ProtectKernelModules' in section 'Service'
Nov 23 21:31:49 ubuntu systemd[1]: [/lib/systemd/system/redis-server.service:32] Unknown lvalue 'ProtectKernelTunables' in section 'Service'
Nov 23 21:31:49 ubuntu systemd[1]: [/lib/systemd/system/redis-server.service:33] Unknown lvalue 'ProtectControlGroups' in section 'Service'
Nov 23 21:31:49 ubuntu systemd[1]: [/lib/systemd/system/redis-server.service:34] Unknown lvalue 'RestrictRealtime' in section 'Service'
Nov 23 21:31:49 ubuntu systemd[1]: [/lib/systemd/system/redis-server.service:35] Unknown lvalue 'RestrictNamespaces' in section 'Service'

P.S. Is there an official place to report bugs in your redis distribution (since everyone seems to be using it)?

Comment From: chrislea

@shravan2x Thanks for pointing this out. 5.0.2 just dropped, and I did some hacky stuff to the relevant bits so that I think this should be kosher now. Meaning, xenial won't have those lvalue entries, but newer distros that support them should. I did some very basic testing, and things seem to be working, but if you could verify that things "just work" as expected I'd appreciate it.

Comment From: shravan2x

@chrislea Yeah, the new package works. It just displays this harmless error, not sure if it's something on your end:

systemd[1]: redis-server.service: PID file /var/run/redis/redis-server.pid not readable (yet?) after start: No such file or directory

I have a few more questions: 1. Could you set up an empty Github repo to use as an issue tracker? You could then link it from your PPA page so people know where to report issues. 2. I see that systemd service entries exist for both redis and redis-server. However, they both seem to point to the same instance. Is this intended?

Comment From: chrislea

@shravan2x Okay, few things here.

The thing with the PID ... I'm honestly not sure. I'll try to take a look at it, but that probably won't happen until (at least) after re:Invent next week. I doubt it will cause any real-world issues.

As for the repo to use as a tracker: I don't want to do this simply because I'm not the "official" source for redis. I've talked to the folks as Redis Labs a bit about this (super good crew over there, by the way), but there isn't any real conclusion as of today. So for now, this is just my personal stab at getting people an easy to use, up-to-date redis install, and I don't want to put myself on the hook for anything past that.

In terms of the dual service entries, I'm not sure what's going on there. I only have a service file for redis-server on the machines I've tested on. What does:

% dpkg -S /lib/systemd/system/redis.service

show?

Comment From: shravan2x

It shows:

dpkg-query: no path found matching pattern /lib/systemd/system/redis.service

But the file is at /etc/systemd/system/redis.service:

lrwxrwxrwx 1 root root   40 Nov 24 05:59 redis.service -> /lib/systemd/system/redis-server.service

Comment From: chrislea

@shravan2x Okay. Well if it's not owned by any package (which is what that response means), then it's not part of the packages I've been pushing out. ¯_(ツ)_/¯

I'd guess that you could just delete that file, or more likely move it somewhere else to make sure nothing breaks, and things should be okay. But I'm not on that system so proceed with care and assume the usual "don't do anything I say unless you're okay with things breaking" kind of situation.

Comment From: shravan2x

@chrislea I just checked and it appears redis.service is created by Redis itself when it's run. That's funny because the service needs to be run once before the service is created.

Either way, it isn't a big deal.

Comment From: kid1412621

9102 now

Comment From: rchintsov

Hello there! Already 2020, end of May. About 1 month after the big release I can't find any 6+ version in any Ubuntu repos, including official 20.04 focal repo and already mentioned ppa:chris-lea/redis-server. Please, create the official PPA with fresh versions.

Comment From: adiospeds

Its June 2020 we are looking at places to download and support our old redis version installed via apt as we move towards upgrading our old instances which are taking time. #RedisOfficiaRepoNeeded :(

Comment From: chrislea

@rchintsov Redis 6.x is in my PPA now just FYI. Sorry for the delay.

Comment From: adiospeds

Hey @chrislea we had pinned redis5:5.0.8-1chl1~xenial1 in our puppet but it seems the package does not exist in your repo any more. Is there a reason you are removing the older version from you repo. We do want to add the latest 5:6.0.5-1chl1~xenial1 to our redis clusters which contain other nodes at version: 5:5.0.8-1chl1~xenial1.

Comment From: chrislea

@adiospeds I don't remove any files, they are still available here. Unfortunately, Launchpad doesn't support pinning, which is why this isn't working for you. I don't have any control over this so unfortunately there's nothing I can do.

Comment From: zippy-zebu

@chrislea Why did you disable Add-support-for-USE_SYSTEM_JEMALLOC-flag.patch in ppa for 6.0.5 for bionic ? Debian enables this patches .

Comment From: chrislea

@zippy-zebu The Redis team asked me to publish binaries with the bundled libraries statically linked.

Comment From: irfanjunaid

Hey @chrislea Facing issues when managing the redis service through Systemd...

Related issues:

7284

7217

Not sure if this fix - 16dbe51 is merged on to 6.0.6 but the above issue is definitely there in the current version - 6.0.5 when I installed from the PPA today. Actually I came across this issue only after noticing the error when trying to restart the redis server.

As usual, configured these: - daemonize is set to no - supervised is set to systemd - Systemd service type set to notify

But when trying to restart the redis, it stays at

Active: activating (start)

and then fails with

systemd[1]: redis.service: Failed with result 'protocol'.

Comment From: readingtype

@irfanjunaid @chrislea

Not sure if this fix - 16dbe51 is merged on to 6.0.6 but the above issue is definitely there in the current version - 6.0.5

I've just pulled the source for 6.0.6 from the PPA, built it on Ubuntu 18.04, and installed. The issue of constant restarting by Systemd is still present. I did install libsystemd-dev before doing the build.

I do not build packages locally very often so I may have missed something, but my unit setup matches the description in #7284 and @irfanjunaid's comment above.

Comment From: yossigo

Hi @chrislea, seems like there's great demand for this and it's been neglected for a long time so I'll be more than happy to push this forward.

Ideally we should have this as automated as possible and integrated with GH Actions and triggered by a new release.

If that's OK with you, I'll just pick up your debian/ packaging stuff as a starting point and figure out what else needs to be done. If you have additional pointers or thoughts, please do let me know. Thanks!

Comment From: chrislea

No problem @yossigo happy to help. Those build sources haven't changed too much in a while so they should be a decent starting point. There are some patches in debian/patches that you might want to look at as they do modify the C code I think. So it might be worth making sure they aren't modifying anything that's likely to change if you're going for full automation.

Another thing I should point out is that I build all of the things in deps statically, which is not the "Debian way" of doing things. Their zen is that you should instead install all those things from distinct packages and link them dynamically and they patch what they ship accordingly. However, the folks at Redis Labs asked me to do it statically so that's what I'm doing.

If you'd like to set up a video chat or anything so I can go into more details I'm happy to. I can also maybe advise on how to build for different distros using sbuild as I've done that in the past. Just let me know.

Comment From: yossigo

@chrislea I've started to look at this, and have some thoughts/questions but no real progress yet. I'll share that here and will be happy to feedback as I'm definitely not up to date with recent (i.e. last decade's) Debian packaging practices.

1. I assume we'll have a dedicated redis-debian repo and will create some workflow around git-buildpackage.
2. Any best practice/tool to use a single source for all Ubuntu (and later on, Debian) builds? For example, automatically adjusting debian/changelog etc.?
3. Breaking this work into smaller steps, I guess that could be something like: a) Getting builds automated (scripts / GitHub Actions) for source and limited set of binary packages. b) Push source packages to have a formal PPA. c) Create a formal repo (to support Debian as well) along with a multi-distro, multi-arch sbuild/pbuilder setup.

Does that make sense?

Comment From: yossigo

I've created the redis-debian repo with some automation to generate and upload source packages for xenial, bionic and focal. All the Debian packaging work is based as-is on @chrislea's work (thank you!).

The PPA is now up at ppa:redislabs/redis, will greatly appreciate any feedback if something is broken/not right. If all goes well, that should be the PPA we'll push new versions to from now on.

Comment From: readingtype

@yossigo This is great to hear. I'll give it a look as soon as I can. Thank you.

I have a question that may or may not be off-topic: is there a chance that RedisJSON could be integrated into the package (or at least included in the PPA as a subsidiary package)? Is there a technnological reason why not?

Comment From: dvergeylen

# Adding the PPA source list:
sudo sh -c 'echo "deb http://ppa.launchpad.net/redislabs/redis/ubuntu focal main" > /etc/apt/sources.list.d/redislabs-ubuntu-redis.list'
# adapt 'focal' to xenial/bionic/focal according to your needs (no other dists at this time of writing)

# Adding the PUBKEY to trust it:
gpg --no-default-keyring --keyring /tmp/redislab-redis.gpg --keyserver keyserver.ubuntu.com --recv-keys CC59E6B43FA6E3CA
sudo sh -c "gpg --no-default-keyring --keyring /tmp/redislab-redis.gpg --export > /etc/apt/trusted.gpg.d/redislab-redis.gpg"
rm /tmp/redislab-redis.gpg

# Update list of packages
sudo apt update

Don't use add-apt-repository ppa:redislabs/redis as it still uses apt-key, which is deprecated (from man apt-key):

apt-key - Deprecated APT key management utility

Comment From: itamarhaber

@readingtype, although off-topic, RedisJSON, and other 3rd party modules will not be in the same package, even if only because of license pollution. That said, Redis Labs' modules packages could wind up in that PPA to enable sane and trivial consumption.

Comment From: yossigo

@dvergeylen I believe you're quoting the instructions provided by launchpad.net, is that correct? As far as I know this text is provided by Launchpad and cannot be changed.

Comment From: dvergeylen

Hi @yossigo ,

Actually I didn't even know the instructions were described on launchpad.net although that seems obvious now. 🙃 add-apt-repository has historically been the de facto tool to add ppa repositories and I was surprised to encounter a warning while adding ppa:redislabs/redis (in FR):

sudo add-apt-repository ppa:redislabs/redis
[...redacted]
gpg: le trousseau local « /tmp/tmpr_vbbs74/pubring.gpg » a été créé
gpg: /tmp/tmpr_vbbs74/trustdb.gpg : base de confiance créée
gpg: clef CC59E6B43FA6E3CA : clef publique « Launchpad PPA for Redis Labs » importée
gpg:       Quantité totale traitée : 1
gpg:                     importées : 1
Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
gpg: aucune donnée OpenPGP valable n'a été trouvée.

so I ended up doing a bit of search to find a proper way to add the repo + its pubkey and document it so it could benefit others. If anyone has a nicer way of doing it it would be even better!

I am afraid there is nothing you can do to update the instructions on launchpad.net as these are primarily for Ubuntu users and apt-key doesn't seem to be deprecated in Ubuntu (at least not in 20.04). Debian started to deprecated apt-key in 2019 "because gnupg has been demoted to a Recommends in apt [...]".

So this might affects Debian users only, not Ubuntu users 🧐.

Comment From: kohenkatz

@dvergeylen there has been work done to make add-apt-repository use gnupg directly, but it seems that it was reverted because it wasn't complete and was causing problems. I'm not on my work computer now where I have links to the source for that, but I'll try to find it later.

Comment From: kohenkatz

@dvergeylen This is a lot more confusing than it originally seemed...

• Source for Ubuntu Focal (20.04 LTS) still uses apt-key. In practice, this means I would expect Ubuntu to support it in 20.04 through April 2025.
• Ubuntu's Git repository also has debian/* branches with upstream code. The debian/buster and debian/sid use the gpg executable directly.

However, if you look at the details in the commit log, you can see that the last commit on the relevant file in that branch was in 2015 and was tagged as 0.96.9debian1. The last commit on the relevant file in the ubuntu/focal branch was in 2016, is version 0.96.24.7, and says this:

Port AptAuth.py from gpg command parsing to apt-key command parsing.

In the Debian software-properties package release notes, it says that the most recent version 0.96.20.2-2 and it includes a bug fix for Debian bug 867681. The bug tracker for that bug says this version is built on Ubuntu's version 0.96.24.32.7-1. Looking further down in the log there, we can see that Ubuntu's version 0.96.24.25 stopped using apt-key and then apparently started to use it again. However, when I look at Ubuntu's version history, it doesn't seem that is the case.

I think we will have to get official word from Debian and Ubuntu package maintainers for software-properties/software-properties-common to find out what is actually going on here.

Comment From: yossigo

Closing this issue now as there's an official PPA. Any issues with it should be filed in the dedicated repository.