Saturday 11 February 2012

Testing Build using Koji

If anyone has been following my posts so far I have created an RPM build that I am currently testing. For this post I am writing about "Koji" which is a client-server system that submits build job to koji's build hub. It is then retrieved by separate server system (builder) which tries to build it resulting in a success or fail!. I have used my gnuchess SRPM package that I have created from previous post for this test.

The first command I used below in bold instructs koji to build the package using the packages that are available in Fedora 17's distro. The "--scratch" option tells koji to build but do not tag it meaning it will not be included in Fedora.
 
koji build f17 --scratch ~/rpmbuild/SRPMS/gnuchess6.0.1-1.fc16.src.rpm
Unable to log in, no authentication methods available

As you can see above, this was the message I got back from trying the koji command on my package. After some searching I found that the problem was that I did not have authentication and so I ran this command to obtain authentication.
  
fedora-packager-setup

Running this command generated a client certificate from the Fedora Account System (FAS). This was the whole output from running that command >>>>>>>>>>

This process required me to enter a FAS account which I had created already and also the password. The output also let me know that the Browser certificate was exported to ~/fedora-browser-cert.pl2

After this had completed, I re-ran the initial koji command that I first tried and this time it went through without any authentication message. The command again was  
koji build f17 --scratch ~/rpmbuild/SRPM/gnuchess6.0.1-1.fc16.src.rpm

This is the screen shot of my terminal as the task was going on and the last line before it ended says that it completed successfully! Also from this we can see that a taskID of 3783537 was given to this task.

With this number we can check to see the status of a task: 
koji watch-task taskID

Where taskID is the number given at the last line after the koji build command completed.

This was was the output from entering the koji watch command on my taskID










The whole process of using koji on my build was fairly straightforward with the exception of the authentication message that I received which required some searching. Overall the process I would say had taken 1hr and a half to get here.

Links
My SRPM of gnuchess
Create a Fedora FAS account 

Monday 6 February 2012

Testing BuildRequires with Mock

Recently working with spec files I've learned BuildRequires are packages required to build the software. In this post I will give my experience using Mock which is used to test a packages BuildRequires. This will help us test that the BuildRequires are complete and accurate without anything missing.

To get started setting up Mock, I installed the mock package with this command:
yum install mock

Then I had to add my username to the mock group:
usermod -aG mock namnguyen

After adding my username to the mock group I ran mock as superuser and received this error message as I did not change to my regular account which I had just added to the mock group.

[root@timeline ~]# mock -r fedora-16-x86_64 ~/rpmbuild/SRPMS/gnuchess-6.0.1-1.fc16.src.rpm
ERROR: mock will not run from the root account (needs an unprivileged uid so it can drop privs)

I then changed to the regular user account that I added to the mock group and was then able to run mock on my SRPM.

[namnguyen@timeline ~]$ mock -r fedora-16-x86_64 ~/rpmbuild/SRPMS/gnuchess-6.0.1-1.fc16.src.rpm
INFO: mock.py version 1.1.19 starting...
State Changed: init plugins
INFO: selinux enabled
State Changed: start
INFO: Start(/home/namnguyen/rpmbuild/SRPMS/gnuchess-6.0.1-1.fc16.src.rpm)  Config(fedora-16-x86_64)
State Changed: lock buildroot
State Changed: clean
INFO: chroot (/var/lib/mock/fedora-16-x86_64) unlocked and deleted
State Changed: unlock buildroot
State Changed: init
State Changed: lock buildroot
Mock Version: 1.1.19
INFO: Mock Version: 1.1.19
INFO: calling preinit hooks
INFO: enabled root cache
INFO: enabled yum cache
State Changed: cleaning yum metadata
INFO: enabled ccache
State Changed: running yum
State Changed: creating cache
State Changed: unlock buildroot
INFO: Installed packages:
State Changed: setup
State Changed: build
INFO: Done(/home/namnguyen/rpmbuild/SRPMS/gnuchess-6.0.1-1.fc16.src.rpm) Config(fedora-16-x86_64) 2 minutes 34 seconds
INFO: Results and/or logs in: /var/lib/mock/fedora-16-x86_64/result
State Changed: end

The first time I had ran this command it hung on "State Changed: running yum" for 10 minutes before I interrupt and exited. Not really sure why it hung there I checked /var/lib/mock/fedora-16-x86_64/result/build.log for any info and results. The only info within build.log was one line that showed Mock Version: 1.1.19. I gave mock another try and this time it continued on past "Stage Changed: running yum" and continued until it was done.

The second attempt had only taken 2 minutes 34 seconds and right now I'm not really sure why mock was not successful on the first attempt but was successful on the second attempt. My experience using mock in this process had taken an hour and a half of which a lot of time was spent on searching for the reason for my issue that I mentioned above.


Sunday 5 February 2012

RPM Writing continued

Continuing on from my last post "RPM Writing", I will be be taking the RPM that was successfully built and testing it with rpmlint command. This command should check for common errors in the spec file,binary RPM and source RPM.

rpmlint on the spec file

[root@timeline ~]# rpmlint -i rpmbuild/SPECS/gnuchess.spec
0 packages and 1 specfiles checked; 0 errors, 0 warnings.

rpmlint on the binary RPM

[root@timeline ~]# rpmlint -i rpmbuild/SRPMS/gnuchess-6.0.1-1.fc16.src.rpm
1 packages and 0 specfiles checked; 0 errors, 0 warnings.

rpmlint on the source RPM

[root@timeline ~]# rpmlint rpmbuild/RPMS/x86_64/gnuchess-6.0.1-1.fc16.x86_64.rpm
gnuchess.x86_64: W: incoherent-version-in-changelog Nguyen ['6.0.1-1.fc16', '6.0.1-1']
gnuchess.x86_64: E: info-files-without-install-info-postin /usr/share/info/gnuchess.info.gz
gnuchess.x86_64: E: info-files-without-install-info-postun /usr/share/info/gnuchess.info.gz
gnuchess.x86_64: W: file-not-utf8 /usr/share/info/gnuchess.info.gz
gnuchess.x86_64: E: info-dir-file /usr/share/info/dir
gnuchess.x86_64: E: info-files-without-install-info-postin /usr/share/info/dir
gnuchess.x86_64: E: info-files-without-install-info-postun /usr/share/info/dir
gnuchess.x86_64: W: no-manual-page-for-binary gnuchessu
gnuchess.x86_64: W: no-manual-page-for-binary gnuchess
gnuchess.x86_64: W: no-manual-page-for-binary gnuchessx
1 packages and 0 specfiles checked; 5 errors, 5 warnings.

After running rpmlint to check for errors in the spec file, binary rpm and source rpm we are able to see what was checked, errors and warnings. There were no errors for gnuchess.spec and gnuchess-6.0.1-1.fc16.src.rpm but there were errors and warnings returned for gnuchess-6.0.1-1.fc16.src.rpm. Now that we know that which file has errors we will need to address them in order to have the rpm conform to rpm guidelines.

We can run the same rpmlint command with -i option to show informative message and additional explanations to the reported message

[root@timeline ~]# rpmlint -i rpmbuild/RPMS/x86_64/gnuchess-6.0.1-1.fc16.x86_64.rpm
gnuchess.x86_64: W: incoherent-version-in-changelog Nguyen ['6.0.1-1.fc16', '6.0.1-1']
The latest entry in %changelog contains a version identifier that is not
coherent with the epoch:version-release tuple of the package.

gnuchess.x86_64: E: info-files-without-install-info-postin /usr/share/info/gnuchess.info.gz
This package contains info files and provides no %post scriptlet containing a
call to install-info.

gnuchess.x86_64: E: info-files-without-install-info-postun /usr/share/info/gnuchess.info.gz
This package contains info files and provides no %postun scriptlet containing
a call to install-info.

gnuchess.x86_64: W: file-not-utf8 /usr/share/info/gnuchess.info.gz
The character encoding of this file is not UTF-8.  Consider converting it in
the specfile's %prep section for example using iconv(1).

gnuchess.x86_64: E: info-dir-file /usr/share/info/dir
You have /usr/info/dir or /usr/share/info/dir in your package. It will cause
conflicts with other packages and thus is not allowed. Please remove it and
rebuild your package.

gnuchess.x86_64: E: info-files-without-install-info-postin /usr/share/info/dir
This package contains info files and provides no %post scriptlet containing a
call to install-info.

gnuchess.x86_64: E: info-files-without-install-info-postun /usr/share/info/dir
This package contains info files and provides no %postun scriptlet containing
a call to install-info.

gnuchess.x86_64: W: no-manual-page-for-binary gnuchessu
Each executable in standard binary directories should have a man page.

gnuchess.x86_64: W: no-manual-page-for-binary gnuchess
Each executable in standard binary directories should have a man page.

gnuchess.x86_64: W: no-manual-page-for-binary gnuchessx
Each executable in standard binary directories should have a man page.

1 packages and 0 specfiles checked; 5 errors, 5 warnings.
[root@timeline ~]#

I find that using the -i option with rpmlint is very helpful as it helps make the errors more human readable so that you can know where to look in order to fix the errors. I found that this site list most of the errors I have which are common so I am able to edit my spec file and make the changes.
common Rpmlint issues - Fedora Project

[root@timeline ~]# rpmlint rpmbuild/RPMS/x86_64/gnuchess-6.0.1-1.fc16.x86_64.rpm
gnuchess.x86_64: E: standard-dir-owned-by-package /usr/share/info
gnuchess.x86_64: W: file-not-utf8 /usr/share/info/gnuchess.info.gz
gnuchess.x86_64: E: info-dir-file /usr/share/info/dir
gnuchess.x86_64: W: no-manual-page-for-binary gnuchessu
gnuchess.x86_64: W: no-manual-page-for-binary gnuchess
gnuchess.x86_64: W: no-manual-page-for-binary gnuchessx
1 packages and 0 specfiles checked; 2 errors, 4 warnings.

Currently still working on trying to address the binary RPM errors as its stands at 2 errors and 4 warnings. Will update this post shortly once they are resolved.

My spec file, RPM and SRPM can be  downloaded from these links I have posted.
gnuchess.spec
gnuchess-6.0.1-1.fc16.x86_64.rpm
gnuchess-6.0.1-1.fc16.src.rpm

RPM Writing

Writing my first RPM was a rough and interesting process which required some effort put into build and testing. For this post I will explain the steps I took to write and build an rpm successfully. The RPM Package Manager (RPM) will contain a compiled version of software, which for me was the gnuchess software package.

There are some preparations we will have to do before we can start building RPM packages.

packages that we will require and need to install using yum command 
yum groupinstall "Fedora Packager"
yum install rpmlint yum-utils

next we will create the ~/rpmbuild directory which also creates six subdirectories within
The command to create ~/rpmbuild will also create ~/.rpmmacros file alongside, entering the one single command will create all this
rpmdev-setuptree

note: To clean out dev directories, we can use command rpmdev-wipetree

Now that we have the required packages installed and directories created we can start building our RPM package!!!

First we will put the source code in the form of a tarball in ~/rpmbuild/SOURCES directory, my tarball was gnuchess-6.0.1.tar.gz

Once this is done, we will change directory to ~/rpmbuild/SPECS to create an empty skeleton spec file. The spec file is essentially important as it defines the RPM we are building by containing metadata and scripting necessary for the build. We will create the skeleton spec file with the command
rpmdev-newspec nameofPackage example of name is gnuchess.spec

We can attempt to build our package right now with the command
rpmbuild -ba gnuchess.spec where -ba is for build all. However it's certain to get several errors as there is nothing written in the spec files eight sections.

I found this part of writing rpm to be the most challenging as using the rpmbuild command countless times had produce errors seen below

error: Installed (but unpackaged) file(s) found:
   /usr/bin/gnuchess
   /usr/bin/gnuchessu
   /usr/bin/gnuchessx
   /usr/share/gnuchess/book.bin
   /usr/share/gnuchess/gnuchess.ini
   /usr/share/info/dir
   /usr/share/info/gnuchess.info.gz

RPM build errors:
    Installed (but unpackaged) file(s) found:
   /usr/bin/gnuchess
   /usr/bin/gnuchessu
   /usr/bin/gnuchessx
   /usr/share/gnuchess/book.bin
   /usr/share/gnuchess/gnuchess.ini
   /usr/share/info/dir
   /usr/share/info/gnuchess.info.gz
 
After a little bit of searching on the web I found the problem to be that I have not included these files in my spec file. The solution I found to this was to include n my %files section of my spec file which was empty

%defattr(-,root,root,-)
%{_bindir}/*
%{_infodir}/*
/usr/share/gnuchess

note: This line will differ depending on the package that you are trying to build

After including these lines I re-ran rpmbuild -ba gnuchess.spec and finally created my RPM successfully as the command did not produce any errors and last line returned an exit status 0. To confirm I checked the RPMS folder for the binary rpm and the SRPMS for the source RPM to confirm the builds success as shown from the print screen of my terminal above.

We have now got a RPM created and in my next post I will be testing it with rpmlint for errors and warnings. This process I would say took me a good couple of hours to get to this stage as it involved some reading on spec file and testing. I have included some links that I found helpful to understand rpm's and also spec files and their sections.

packaging software with RPM from ibm.com 

how to create Spec file from fedoraproject.org

creating spec file from rpm.org