HOWTO: Initial Setup of slackrepo (plus some questions)
by bassmadrigal from LinuxQuestions.org on (#21CM4)
==BACKGROUND==
I've been a big fan of David Spencer's (55020) slackrepo. It seemed like a great project and an awesome idea for some running multiple Slackware machines that might need some overlapping packages. I'd been interested in running it for a long time, but never went through the process of setting it up because I was relatively happy with sbopkg (even though it is an absolute pain to rebuild the needed packages when updates are available on SBo. However, I've been gearing up for upgrading my main machine to 14.2, but I didn't want to be left for a long time without my needed 3rd-party programs, and my desktop isn't the fastest right now, so it takes a LONG time to build programs, and when I have 210 SBo packages, I couldn't have that length of downtime after upgrading while waiting for programs to build. So, I decided to host a 14.2 VM on my HTPC to build programs in the background which will be easily transferable to my desktop. Initially, I was just using sbopkg to build programs, but I thought this is probably the perfect time to take the dive into slackrepo and get it set up. Then I can just use slackpkg+ to install everything needed onto my desktop. Getting this set up was more work than I imagined... here's my details, including some follow-up questions. Hopefully this can help others who are trying to set it up. If this is liked well enough, I can add it to the SlackWiki (or someone else is free to take it and do the same -- making changes as they see fit).
==HOWTO==
===INSTALLATION===
First thing to do is head to the slackrepo site and download the pre-built package and install it.
Code:wget https://dl.dropboxusercontent.com/u/26560685/slackrepo-0.2.0rc1-noarch-1_dbs.txz
installpkg slackrepo-0.2.0rc1-noarch-1_dbs.txzThe easy part is done. You can now start building programs.
Code:slackpkg build packagenameThe first time you use the program to build something, it will set up the folder structure and sync with the SBo repo automatically. slackrepo will work out dependencies based on the REQUIRES line in the .info files of the packages.
===SLACKPKG+ SETUP===
However, this is only a small portion of the capabilities of this program. My next goal was to get a slackpkg+ repo enabled. This took a large amount of time as I had never done anything like it before. If you're familiar with it, you'd probably breeze through this section.
First thing I did after trying a few test compiles is look into the configs for this. There resides a different config for various pre-installed repos: SBo, ponce (SBo branch used for -current), csb (willysr's Cinnamon SlackBuilds), and msb (willysr's Cinnamon SlackBuilds). By default, slackrepo will use SBo, so we'll be looking into that one.
====GPG KEY CREATION====
As a bit of a pre-requisite, we're going to generate a GPG key so we can properly sign our packages. Normally, you simply do this using gpg --gen-key and follow the prompts, however, since I'm running this on a VM, there's a snag because it doesn't generate the needed entropy to create a random key. So I had to do this on my desktop and then move the key to the VM.
So, once you log into an actual computer and not a VM, you can run gpg --gen-key to start the setup wizard. Most likely, it's good enough to use the defaults (at least, that's what I did). Make note of your "User ID" (a combination of your real name, comment and email address). This will be used in the slackrepo config later on.
Once your key is generated, we need to move it to the VM. First, we'll package them up into an encrypted file, then move it over.
First we need to find out what your key ID is. To do this, run:
Code:gpg -K(If you're running slackrepo on a real machine and not a VM, you're now done with the GPG process)
We're looking for your 8 character key ID under the sec line.
Code:sec 2048R/1234ABCD 2016-11-12To make this easier for copy/paste, store your key in a variable (replacing the 1234ABCD with your key ID).
Code:KEYID=1234ABCDNow, we need to export your public key.
Code:gpg --output pubkey.gpg --export $KEYIDNow, we're going to export your private key, however, combine it with the public key and to keep it secure we're going to encrypt it. When it asks for a passphrase, use a strong one, but don't forget it (you only need it once).
Code:gpg --output - --export-secret-key $KEYID |\
cat pubkey.gpg - |\
gpg --armor --output keys.asc --symmetric --cipher-algo AES256Now we have a keys.asc file that we just need to transfer to the VM. You can do this anyway you see fit. Since it is encrypted, you can transfer it without additional security. I just temporarily hosted mine on my apache install and then used wget to download it on my VM.
Once you have the file on your VM, you need to import it.
Code:gpg --no-use-agent --output - keys.asc | gpg --importYou now have your GPG key setup on your VM. (It's probably a good idea to remove the keys.asc file from your various machines so nobody can get their hands on it and try and crack it -- which would not be an easy task, but it's still better to not leave it laying around.)
====OTHER MINOR PRE-REQS====
If you're going to want to serve packages for slackpkg+, you'll need to have apache up and running. To do this, you simply need to add the execute bit to /etc/rc.d/rc.httpd and start it.
Code:chmod +x /etc/rc.d/rc.httpd
/etc/rc.d/rc.httpd startOnce this is setup, we'll need to symlink the package directory into /var/www/htdoc/ so your webserver can actually make the packages available.
Code:mkdir -p /var/www/htdocs/pkgrepo/SBo/
ln -s /var/lib/slackrepo/SBo/packages/14.2 /var/www/htdocs/pkgrepo/SBo/If you want to support an RSS feed, you'll need to create a UUID by simply running:
Code:uuidgen -tMake note of this as well because you'll need it when editing the conf file.
====SLACKREPO CONF SETUP====
Now we just need to use your favorite editor to open up /etc/slackrepo/slackrepo_SBo.conf so we can get the slackpkg+ portion set up.
First thing we need to do is enable the USE_GENREPOS on line 95. You do this by changing the 0 to a 1.
Next, we need add your GPG user ID to the REPOSOWNER variable (you did save that, right? If you didn't you can get it again from the uid portion of gpg -K).
Now we need to adjust the DL_URL from localhost to your local IP (probably not needed)
And finally, if you want to provide an RSS feed, you'll need to use that UUID you saved from earlier and add it under RSS_UUID
Code:RSS_UUID="2d88e7b4-a91c-11e6-9ba0-08002775c17a"You're finally done. Now, the next time you build a package, it will generate all the needed files for slackpkg+ to work.
Code:================================================================================
gen_repos_files.sh 21:39:42
================================================================================
# -------------------------------------------------------------------#
# $Id: gen_repos_files.sh,v 1.92 2014/07/31 20:27:53 root Exp root $ #
# -------------------------------------------------------------------#
--- Generating repository metadata for /var/lib/slackrepo/SBo/packages/14.2/x86_64 ---
--- Repository owner is Jeremy Brent Hansen (bassmadrigal) <jebrhansen+SBo@gmail.com> ---
Enter your GPG passphrase: .
Generating a GPG-KEY file in '/var/lib/slackrepo/SBo/packages/14.2/x86_64',
containing the public key information for 'Jeremy Brent Hansen (bassmadrigal) <jebrhansen+SBo@gmail.com>'...
--> Generating .asc file for mutagen-1.35-x86_64-1_SBo.tgz
--> Generating .asc file for chromaprint-1.2-x86_64-1_SBo.tgz
--> Generating .asc file for dos2unix-7.3.4-x86_64-1_SBo.tgz
--> Generating .asc file for picard-plugins-20160206-x86_64-1_SBo.tgz
--> Generating .asc file for picard-1.3.2-x86_64-1_SBo.tgz
--> Generating .asc file for transmission-2.92-x86_64-1_SBo.tgz
--> Generating .asc file for youtube-dl-2016.11.04-x86_64-1_SBo.tgz
--> Generating .asc file for etckeeper-1.18.5-x86_64-1_SBo.tgz
Finished gen_repos_files.sh at 21:39:49Hopefully I haven't butchered anything too bad in my guide, but if I have, please let me know and I'll adjust it.
==QUESTIONS==
Now, even after all of this, I do still have questions. These are open to the community to answer if you're aware or maybe I'll have to wait for David to see this.
Overall, I'm extremely impressed with this so far. It is really powerful and impressive. David, you've done an amazing job!
I've been a big fan of David Spencer's (55020) slackrepo. It seemed like a great project and an awesome idea for some running multiple Slackware machines that might need some overlapping packages. I'd been interested in running it for a long time, but never went through the process of setting it up because I was relatively happy with sbopkg (even though it is an absolute pain to rebuild the needed packages when updates are available on SBo. However, I've been gearing up for upgrading my main machine to 14.2, but I didn't want to be left for a long time without my needed 3rd-party programs, and my desktop isn't the fastest right now, so it takes a LONG time to build programs, and when I have 210 SBo packages, I couldn't have that length of downtime after upgrading while waiting for programs to build. So, I decided to host a 14.2 VM on my HTPC to build programs in the background which will be easily transferable to my desktop. Initially, I was just using sbopkg to build programs, but I thought this is probably the perfect time to take the dive into slackrepo and get it set up. Then I can just use slackpkg+ to install everything needed onto my desktop. Getting this set up was more work than I imagined... here's my details, including some follow-up questions. Hopefully this can help others who are trying to set it up. If this is liked well enough, I can add it to the SlackWiki (or someone else is free to take it and do the same -- making changes as they see fit).
==HOWTO==
===INSTALLATION===
First thing to do is head to the slackrepo site and download the pre-built package and install it.
Code:wget https://dl.dropboxusercontent.com/u/26560685/slackrepo-0.2.0rc1-noarch-1_dbs.txz
installpkg slackrepo-0.2.0rc1-noarch-1_dbs.txzThe easy part is done. You can now start building programs.
Code:slackpkg build packagenameThe first time you use the program to build something, it will set up the folder structure and sync with the SBo repo automatically. slackrepo will work out dependencies based on the REQUIRES line in the .info files of the packages.
===SLACKPKG+ SETUP===
However, this is only a small portion of the capabilities of this program. My next goal was to get a slackpkg+ repo enabled. This took a large amount of time as I had never done anything like it before. If you're familiar with it, you'd probably breeze through this section.
First thing I did after trying a few test compiles is look into the configs for this. There resides a different config for various pre-installed repos: SBo, ponce (SBo branch used for -current), csb (willysr's Cinnamon SlackBuilds), and msb (willysr's Cinnamon SlackBuilds). By default, slackrepo will use SBo, so we'll be looking into that one.
====GPG KEY CREATION====
As a bit of a pre-requisite, we're going to generate a GPG key so we can properly sign our packages. Normally, you simply do this using gpg --gen-key and follow the prompts, however, since I'm running this on a VM, there's a snag because it doesn't generate the needed entropy to create a random key. So I had to do this on my desktop and then move the key to the VM.
So, once you log into an actual computer and not a VM, you can run gpg --gen-key to start the setup wizard. Most likely, it's good enough to use the defaults (at least, that's what I did). Make note of your "User ID" (a combination of your real name, comment and email address). This will be used in the slackrepo config later on.
Once your key is generated, we need to move it to the VM. First, we'll package them up into an encrypted file, then move it over.
First we need to find out what your key ID is. To do this, run:
Code:gpg -K(If you're running slackrepo on a real machine and not a VM, you're now done with the GPG process)
We're looking for your 8 character key ID under the sec line.
Code:sec 2048R/1234ABCD 2016-11-12To make this easier for copy/paste, store your key in a variable (replacing the 1234ABCD with your key ID).
Code:KEYID=1234ABCDNow, we need to export your public key.
Code:gpg --output pubkey.gpg --export $KEYIDNow, we're going to export your private key, however, combine it with the public key and to keep it secure we're going to encrypt it. When it asks for a passphrase, use a strong one, but don't forget it (you only need it once).
Code:gpg --output - --export-secret-key $KEYID |\
cat pubkey.gpg - |\
gpg --armor --output keys.asc --symmetric --cipher-algo AES256Now we have a keys.asc file that we just need to transfer to the VM. You can do this anyway you see fit. Since it is encrypted, you can transfer it without additional security. I just temporarily hosted mine on my apache install and then used wget to download it on my VM.
Once you have the file on your VM, you need to import it.
Code:gpg --no-use-agent --output - keys.asc | gpg --importYou now have your GPG key setup on your VM. (It's probably a good idea to remove the keys.asc file from your various machines so nobody can get their hands on it and try and crack it -- which would not be an easy task, but it's still better to not leave it laying around.)
====OTHER MINOR PRE-REQS====
If you're going to want to serve packages for slackpkg+, you'll need to have apache up and running. To do this, you simply need to add the execute bit to /etc/rc.d/rc.httpd and start it.
Code:chmod +x /etc/rc.d/rc.httpd
/etc/rc.d/rc.httpd startOnce this is setup, we'll need to symlink the package directory into /var/www/htdoc/ so your webserver can actually make the packages available.
Code:mkdir -p /var/www/htdocs/pkgrepo/SBo/
ln -s /var/lib/slackrepo/SBo/packages/14.2 /var/www/htdocs/pkgrepo/SBo/If you want to support an RSS feed, you'll need to create a UUID by simply running:
Code:uuidgen -tMake note of this as well because you'll need it when editing the conf file.
====SLACKREPO CONF SETUP====
Now we just need to use your favorite editor to open up /etc/slackrepo/slackrepo_SBo.conf so we can get the slackpkg+ portion set up.
First thing we need to do is enable the USE_GENREPOS on line 95. You do this by changing the 0 to a 1.
Next, we need add your GPG user ID to the REPOSOWNER variable (you did save that, right? If you didn't you can get it again from the uid portion of gpg -K).
Now we need to adjust the DL_URL from localhost to your local IP (probably not needed)
And finally, if you want to provide an RSS feed, you'll need to use that UUID you saved from earlier and add it under RSS_UUID
Code:RSS_UUID="2d88e7b4-a91c-11e6-9ba0-08002775c17a"You're finally done. Now, the next time you build a package, it will generate all the needed files for slackpkg+ to work.
Code:================================================================================
gen_repos_files.sh 21:39:42
================================================================================
# -------------------------------------------------------------------#
# $Id: gen_repos_files.sh,v 1.92 2014/07/31 20:27:53 root Exp root $ #
# -------------------------------------------------------------------#
--- Generating repository metadata for /var/lib/slackrepo/SBo/packages/14.2/x86_64 ---
--- Repository owner is Jeremy Brent Hansen (bassmadrigal) <jebrhansen+SBo@gmail.com> ---
Enter your GPG passphrase: .
Generating a GPG-KEY file in '/var/lib/slackrepo/SBo/packages/14.2/x86_64',
containing the public key information for 'Jeremy Brent Hansen (bassmadrigal) <jebrhansen+SBo@gmail.com>'...
--> Generating .asc file for mutagen-1.35-x86_64-1_SBo.tgz
--> Generating .asc file for chromaprint-1.2-x86_64-1_SBo.tgz
--> Generating .asc file for dos2unix-7.3.4-x86_64-1_SBo.tgz
--> Generating .asc file for picard-plugins-20160206-x86_64-1_SBo.tgz
--> Generating .asc file for picard-1.3.2-x86_64-1_SBo.tgz
--> Generating .asc file for transmission-2.92-x86_64-1_SBo.tgz
--> Generating .asc file for youtube-dl-2016.11.04-x86_64-1_SBo.tgz
--> Generating .asc file for etckeeper-1.18.5-x86_64-1_SBo.tgz
Finished gen_repos_files.sh at 21:39:49Hopefully I haven't butchered anything too bad in my guide, but if I have, please let me know and I'll adjust it.
==QUESTIONS==
Now, even after all of this, I do still have questions. These are open to the community to answer if you're aware or maybe I'll have to wait for David to see this.
- By default, it includes a number of hintfiles, but they're in a 14.1/ directory. I assume these were designed to work with only the 14.1 repo and not 14.2, so it may or may not work, right? Are there any places where newer hintfiles are available or do you just need to tailor the 14.1 files to a 14.2 install or create your own?
- When I built transmission, it notified me that some files in the chroot were modified and it discarded the changes. Is there a way to view the changes? They were pre-installed files that were a part of a FULL Slackware install, so they weren't new files created by transmission. I'm just wondering what was actually changed.
- Is there a way to have it build a number of packages at once? I just found you can simply do slackrepo build package1 package2 package3 and it will build each in it's own chroot (any dependencies will be built under the chroot of program that depends on them).
- Is it possible to have multiple repos in the same package directory? My main use case for this would be any SlackBuilds I have locally (that either haven't been submitted to SBo or can't be for some reason). Some of those packages have requirements that exist on SBo but that I don't have locally.
- What's the best way to modify .SlackBuild or .info files when hintfiles aren't enough? Is there a way to keep changes when syncs are accomplished with SBo? My use case for this is removing the jdk requirement from kodi's REQUIRES line, because I use Eric's openjdk package and I don't want slackrepo to try and (unsuccessfully) build jdk.
- Is there a way to regenerate the slackpkg+ repo without building a program? I built several packages before I set up the slackpkg+ portion, but I couldn't find any way to have it create the repo without building another program.
- Is there a way to view the READMEs within the program or do you need to manually view them (either through SBo or using a text viewer in the shell)? It'd be nice to be able to have the READMEs popup so I can decide if I should make or modify a hintfile. It's not a big deal if this doesn't exist in the program.
Overall, I'm extremely impressed with this so far. It is really powerful and impressive. David, you've done an amazing job!