This is related to https://github.com/redis/redis/issues/11453
I distribute a set of Windows binaries for Redis with publicly auditable build instructions. The goal is to allow Redis modules to be developed on Windows and then deployed to Linux. That development model has worked well for me, especially because the C + ASM debuggers on Windows are much easier/better (IMHO) than what is available on Linux, and regardless the rest of my development is on Windows.
The current recommendation on https://redis.io/docs/getting-started/installation/install-redis-on-windows/ is to install WSL2, which is a non-ideal development workflow for many Windows developers. A link to the binaries would be great for Windows developers, and I can change the verbiage on that page to suit what Redis needs.
Thanks.
Comment From: oranagra
what do you mean windows binaries for Redis? you mean WSL2 or native Windows? Redis doesn't support windows (not even in source form), so we'll surely won't distribute binaries for it. Besides, for the most part we don't even distribute Linux binaries (we only offer source download). The exception to that is PPA/APT/Snap which we recently started owning (in the past it was handled by others).
Comment From: jonahbeckford
Redis doesn't support windows (not even in source form), so we'll surely won't distribute binaries for it. Besides, for the most part we don't even distribute Linux binaries (we only offer source download).
Oops! I forgot to add the actual link: [removed by matolson. Don't want anyone clicking the link.]
I am suggesting that there be a link to that page for Windows users. Yes, Redis has no official support for Windows. But the code does support being compiled on Windows using MSYS2. No patches to Redis are necessary.
what do you mean windows binaries for Redis? you mean WSL2 or native Windows?
Native Windows
Comment From: madolson
We don't support windows, and can't endorse an arbitrary third party distribution of redis on windows. If it works for you, that's great, but we can't host it.
Comment From: jonahbeckford
The part about endorsing an arbitrary [...] distribution is quite understandable. The part about we can't host it ... when the title (etc.) asks about "adding link" ... is a head scratcher.
Likely something else is going on. No problem. Closing ticket.
If someone else stumbles on this issue, you can build Redis on Windows with MSYS2. Use make -C src redis-server redis-sentinel redis-cli redis-check-rdb redis-check-aof redis-benchmark MALLOC=libc BUILD_TLS=yes CFLAGS=-D_GNU_SOURCE, after installing the prereq packages with pacman -Sy make gcc openssl-devel pkg-config zip msys/msys2-runtime-devel. The tests will not work though.
Comment From: oranagra
@jonahbeckford nothing else is going on... we don't want to distribute or endorse binaries that we (and others) can't validate what's inside of them (may contain malware).
source code or instructions on how to build it, is another thing, and maybe we can add these instructions to the readme or docs. the two lines you posted seems quite straight forward. if there are any prerequisites and other tools that should be installed for that to work, please list them.
@itamarhaber please comment.
Comment From: oranagra
@jonahbeckford you say that it works for you.. what exactly works? * does BGSAVE works? * does replication works? * does the test suite pass?
Comment From: jonahbeckford
- does BGSAVE works?
- does replication works?
- does the test suite pass?
I use Windows only for debugging Redis modules and simple integration tests. Anything production-like (ex. replication) I'll do on Linux.
The test suite does not pass. I'll see if I can capture some pass/fail logs and attach them (might be one or two days before I get the bandwidth). Hopefully that will answer your questions in detail.
Comment From: oranagra
i'm not sure if fork is supported by MSYS2, if it doesn't then BGSAVE and replication have no chance.
passing the tests probably has other challenges (besides a fully functional redis) since they also require kill, ps, and other unix utilities.
in any case, if redis isn't fully functional, i don't see why we would want to recommend that approach, especially considering that WSL2 exists.
p.s. If redis is was fully functional, and all that's needed are some minor makefile and testsuite tweaks, we might have accepted them (like we do for Solaris, i.e. support comes from the community, and we don't have a CI job)
Comment From: jonahbeckford
(I'm running significantly behind. Still need to get the test apparatus installed on Windows, and run the test suite)
i'm not sure if fork is supported by MSYS2
It is supported; old-school Unix way. https://www.cygwin.com/faq.html#faq.api.fork. But let's wait for the test suite.
they also require kill, ps, and other unix utilities
MSYS2 provides most modern unix utilities. I think the real problem I ran into before was the use of (ancient!) tcsh in the test suite. It is possible that the test suite is not runnable without significant investment (which I'm not going to do).
But even macOS does not fully work with the test suite! See my notes at https://gitlab.com/diskuv-ocaml/distributions/dkml-redis/-/blob/23c457af97f35719814a15931be81a0ea457faf4/ci/build-test.sh#L154-167
in any case, if redis isn't fully functional, i don't see why we would want to recommend that approach, especially considering that WSL2 exists.
- If a Windows link or something equivalent for the Redis community isn't available, how will Windows developers know where to contribute patches to get it fully functional?
- I strongly suspect if you ask Windows developers, the majority would prefer a mostly functional Redis rather than WSL2, if for nothing other than basic integration tests. Evidence? Google for the effort others have expended getting Redis working on native Windows.
I hope the expectation is not that I personally develop a fully functional Redis Windows version for you. I'm time-boxing this, especially if there is no desire to support basic mechanisms (ex. a link to a git repository, maybe owned by Redis) to allow others to contribute.
One more thing Redis may consider as a potential negative; better to know earlier rather than later. MSYS2, and its ancestor Cygwin, have a LGPL3 license (https://cygwin.com/licensing.html).
Comment From: oranagra
It is supported; old-school Unix way. https://www.cygwin.com/faq.html#faq.api.fork. But let's wait for the test suite.
Ok, without copy-on-write it'll be slow and memory inefficient, but at least it'll work for development testing.
MSYS2 provides most modern unix utilities. I think the real problem I ran into before was the use of (ancient!)
tcshin the test suite. It is possible that the test suite is not runnable without significant investment (which I'm not going to do).
if we wanna know redis really works, that's something we'll have to resolve
But even macOS does not fully work with the test suite! See my notes at https://gitlab.com/diskuv-ocaml/distributions/dkml-redis/-/blob/23c457af97f35719814a15931be81a0ea457faf4/ci/build-test.sh#L154-167
that's not true, the tests you skipped are failing due to machine slowness, and can be skipped with --no-latency.
it is true that there are a few adjustments in the test suite (and in redis) in order to support MacOS though.
in any case, if redis isn't fully functional, i don't see why we would want to recommend that approach, especially considering that WSL2 exists.
- If a Windows link or something equivalent for the Redis community isn't available, how will Windows developers know where to contribute patches to get it fully functional?
that's a chicken and an egg issue, i would say that someone will need to invest a lot of work to get the initial version working before we would present it as "working", and expect incremental improvements. but as i might have mentioned earlier (and was already mentioned in many other issues in this repo), we do not intend to support Windows since (unlike MacOS and Solaris), the gap seems to be too high, and we won't not mess up the code with hundreds or thousands of patches.
I hope the expectation is not that I personally develop a fully functional Redis Windows version for you. I'm time-boxing this, especially if there is no desire to support basic mechanisms (ex. a link to a git repository, maybe owned by Redis) to allow others to contribute.
IIRC you were the one who initially asked that we distribute binaries you built (suggesting that it works), now you say the gap may be huge? we do not have any desire to maintain that support (other than publishing instructions and maybe accepting a few minor patches). if the effort is significant, someone may decide to create a fork do that there.
the bottom line is that if you can provide instructions and some reassurance that they produce a fully functional redis, we'll be happy to document them. if it requires a few minor fixes to the code, we can merge them. but we don't want to invest much of our time adjusting redis to Windows (since it is not our target platform), we do not want to mess up the code a lot for that, and we do not want to mislead people by providing redis that doesn't fully work, and then face bug reports. and we do.
One more thing Redis may consider as a potential negative; better to know earlier rather than later. MSYS2, and its ancestor Cygwin, have a LGPL3 license (https://cygwin.com/licensing.html).
i'm not sure how this affects us? we do not intend to ship any code or binaries anyway, just source code and instructions.
Comment From: jonahbeckford
IIRC you were the one who initially asked that we distribute binaries you built (suggesting that it works), now you say the gap may be huge?
That is jaw-dropping considering what the title of this issue is: "Consider adding link to pre-compiled Windows binaries". I will gladly send you a $25 Starbucks gift card if a reasonable person can point to where I asked "we (Redis) distribute the binaries". And another $25 Starbucks gift card where I "suggest[ing] that it works" without any qualification about the limited goals or its lack of test suite. And another $25 Starbucks gift card where I said "the gap may be huge".
the bottom line is that if you can provide instructions and some reassurance that they produce a fully functional redis, we'll be happy to document them. if it requires a few minor fixes to the code, we can merge them. but we don't want to invest much of our time adjusting redis to Windows (since it is not our target platform), we do not want to mess up the code a lot for that, and we do not want to mislead people by providing redis that doesn't fully work, and then face bug reports. and we do.
How would a link to an external open-source repository where binaries can be downloaded, PRs can be submitted, and bugs can be filed ... how would that result in you facing bug reports?
But I don't want to lose the important point you mentioned: "we do not want to mess up the code a lot for that (Windows)". That makes a ton of sense. The implication is that Redis may not accept PRs for Windows. That is a showstopper to me (and I think many other sane people as well). Absolutely no point documenting steps for Windows devs if we'll just end up being second-class citizens.
Thanks. Have a good day.
PS. You have my contact info if you can find a reasonable person (that's a legal term, not an indictment) who agrees I said any of those things, and you want those gift cards.
Comment From: oranagra
i'm sorry if i was blunt i didn't mean to offend you.
i do believe you posted a link to binaries which Madelyn removed, and to me that seems to suggest it is working.
Truth be told i didn't follow that link so i don't know what it contains, if it lists any caveats about what it should be used for and which portions are tested to be working.
Please note that it's not the first time people post links to "redis binaries for windows" in issues in this repo, which i suspect were malware in many cases (we always removed the link and asked the poster some questions, we didn't get any response back)
i also think i misunderstood your ask about "developers know where to contribute patches". i was thinking you mean these will be submitted here (we do get quite a lot of bug reports here about redis crashes on Windows, which we don't maintain, so i don't even know what they were running). i'm sorry about my misinterpretation of your text..
If your aim is to maintain an open source port of redis for windows in github (which does happen to also include binaries), and hope to gather other contributors to help collaborate on that, then i think that's an admirable aim. but sadly, i don't think we can really endorse that, like we don't endorse other redis ports.
The fact is that despite Windows being so popular, redis is designed for Linux / POSIX and it never intended to support Windows, considering the gap for proper Windows support is huge, it was always left out of scope intentionally.
However, as i said, if all that's needed are a few small changes to the source code, Makefile, and some simple set of instructions, to support something usable, we could maybe accept these contributions to this repo (like we do support Solaris, as a community effort, even though we don't test it ourselves). But it will be hard to justify that if we don't have a reassurance that it's working (which i believe we do for WSL2)
Comment From: hashimaziz1
The part about endorsing an arbitrary [...] distribution is quite understandable. The part about we can't host it ... when the title (etc.) asks about "adding link" ... is a head scratcher.
Likely something else is going on. No problem. Closing ticket.
If someone else stumbles on this issue, you can build Redis on Windows with MSYS2. Use
make -C src redis-server redis-sentinel redis-cli redis-check-rdb redis-check-aof redis-benchmark MALLOC=libc BUILD_TLS=yes CFLAGS=-D_GNU_SOURCE, after installing the prereq packages withpacman -Sy make gcc openssl-devel pkg-config zip msys/msys2-runtime-devel. The tests will not work though.
Unfortunately this currently fails with the same build error that Cygwin does.
I second that Redis should really have some sort of binary for Windows, it's crazy to me that it doesn't considering Windows is the most common development OS in the world and only set to rise higher - it's like only releasing production binaries for Windows platforms. WSL2 is still too slow and way too buggy to be reliable for even a dev server, I've just spent the entire day purging it from my system several times in an unsuccessful attempt to fix a prevalent bug that stops it from being installed on the latest version of Windows 10. It's also way too Windows-like for a development binary that needs to match production Linux as much as possible, Cygwin and MSYS2 are perfect at doing this like they always have, they just need the extra work to be supported.
Right now it looks like I'll be forced to look at the many third-party binaries, which is terrible for security and will be even more so when the binaries inevitably become unmaintained or out of date in less than a year, as they always historically have. I really hope this situation changes soon and the Redis devs finally decide to fully embrace the majority of developers worldwide who develop on Windows.
Comment From: hashimaziz1
I decided to revisit compiling from source in Cygwin and successfully managed to build all the binaries once I'd removed -Werror from the hiredis dependency - unfortunately trying to run redis-server throws a Segmentation fault (core dumped) error, which seems like a problem that can only be fixed in the source.
Here's a copy-pastable Bash command of the process if anyone wants to try and get further:
wget https://download.redis.io/redis-stable.tar.gz &&
tar -xzvf redis-stable.tar.gz && cd redis-stable/deps/hiredis &&
sed -ie 's/-Werror //g' Makefile &&
make CFLAGS=-I/usr/include -Wall -Wextra -Werror &&
cd ../fpconv &&
make CFLAGS=-I/usr/include &&
cd ../hdr_histogram &&
make CFLAGS=-I/usr/include &&
cd ../linenoise &&
make CFLAGS=-I/usr/include &&
cd ../../ && make CFLAGS=-I/usr/include LDFLAGS=-L/deps/ &&
make install