Блог:ROSA Planet
A technical blog of ROSA Laboratory.
All content is published under Creative Commons Attribution-ShareAlike 3.0 License (CC-BY-SA)
Please, subscribe to RSS/Atom feed. If you have any questions do not hesitate to contact us
RaceHound 1.0 - Once More about the Races in the Kernel
Version 1.0 of RaceHound system has been released recently. RaceHound, as its name implies allows to detect the so called data races in the Linux kernel.
To put it simple, «data race» is a situation when two or more threads of execution (in an application, in the kernel, etc.) may access the same data in memory concurrently and at least one of the threads changes these data. Such conditions may lead to very unpleasant consequences but are often quite hard to detect.
The tools like KernelStrider may help reveal the races. They usually find a lot of potentials but may produce false alarms too. That is, they may report a potential race when a race is not possible.
On the other hand, RaceHound may miss something but if it has detected a race, the race does happen. By the way, these tools work together very well: KernelStrider finds potential races while RaceHound checks if these races really happen.
This was used not long ago to detect an interesting race in «uvcvideo» driver (webcam support) in the kernel 4.1-rc5. The developers of the driver, however, suggest that nothing bad should happen because of that race, but still.
The ideas implemented in RaceHound are rather simple.
- Place software breakpoints (similar to what the debuggers do) on the instructions that may be involved in the races.
- When a software breakpoint hits, determine the address and the size of the memory area the instruction is about to access.
- Place hardware breakpoints on that memory area to detect appropriate accesses to it.
- Make a delay. If some code tries to access to that memory area during the delay, the hardware breakpoints will trigger and the race will be revealed.
- Remove the hardware breakpoints and let the original instruction execute as usual.
As it is often the case, the devil is in the details. Implementing that algorithm was far from easy. It is no surprise that RaceHound is being developed since the summer of 2012.
Compared to the previous versions (0.x), our specialists overhauled the core components of RaceHound in version 1.0. It was only possible before to analyze one kernel module at a time, with additional restrictions. Now RaceHound is able to monitor the code from several modules and the kernel proper at the same time, any code a software breakpoint can be placed to.
Besides, during the development of RaceHound 1.0, an error in the implementation of the software breakpoints (so called, Kprobes, to be exact) was found and fixed in the kernel. The fix should appear in the kernel 4.1.
ROSA Enterprise Desktop X2
ROSA company is glad to announce ROSA Enterprise Desktop X2 — new version of operating system for enterprise use. The distribution includes software for all common tasks: office suite, Internet browser, E-mail and IM clients, audio and video viewers and editors and a lot of other programs. The distribution is based on ROSA Desktop Fresh KDE but provides additional stability guarantees — all updates except fixes for crucial bugs and security holes are published to RED repositories only after careful testing by QA team and users of ROSA Desktop Fresh.
If you would like to buy this distribution or get ISO image for internal tests, please contact ROSA sales department. See our site for details.
ROSA Fresh Desktop R5 LXDE
As many English-speaking users already noticed, more than a month ago we have published ROSA Desktop Fresh R5 LXDE images on our mirrors but still didn't make the announcement. We are extremely sorry for this delay and now would like to provide you some details about that release.
- Foreword
LXDE, that is Lightweight X11 Desktop Environment is one of the most popular lightweight desktop environments.
In the world of user interfaces lightweight implies
- Instant reaction of GUI applications to user actions and low resource consumption. One of the way to teach this is to drop all potentially slow components such as interpreted languages with dynamic garbage collectors and use compiled programs only (at least in the core system part) and use the same set of shared libraries for all core components - window manager, control center, file manager, taskbar and other desktop items. The latter usually means that all programs with GUI should be implemented using the same GUI toolkit - Qt, Gtk or other. And it would be better if all programs in the system use this toolkit.
- Simplicity of UI concept. To be sure, we have the simplest desktops such as twm or fvmm, but most users would prefer more intuitive desktops that look similar to Windows 95/XP with the task bar at the bottom and the "Start" button at its left side.
If you like the ideas above, then you should definitely try LXDE which tries to implement this lightweight concept:
- It uses a light window manager named OpenBox which misses modern 3D effects but is able to work on almost any video card.
- Its UI concept is similar to the one of Windows 95/XP and is comfortable for users who don't want to become familiar with modern designs such as Gnome3 or MacOS X and at the same time don't want to spend much time on reconfiguring desktop according to their wishes (like it often happens with KDE which can be made to look similar to old-style Windows, but this definitely requires some efforts).
Besides the user interface, one should remember that LXDE doesn't consume much resources so you can install it on your old granny's notebook and still launch Chromium or ROSA Media Player there.
- Download
The LXDE R5 images can be downloaded here:
- System Requirements
- 256 Mb of RAM (512 Mb is recommended for the installed system and 384 Mb for the Live mode).
- 6 Gb of free space at your hard drive
- At least Pentium4/Celeron CPU
- Release Notes
LXDE edition shares the package base with KDE and Gnome, so improvements in one flavor automatically appears in the others (if applicable).
In comparison with the previous release, the following changes appeared in ROSA Desktop Fresh R5:
- Boot loader and installer now supports Secure Boot and have a lot of other improvements to support modern hardware.
- Updated versions of major system components (Linux kernel, Mesa and X11 stack, NVidia and AMD proprietary drivers) as well as user-level software (new Chromium, Thundebird, Firefox and others).
- Automated launch of TRIM for SSD (once a week), implemented by means of ssd-utils package.
- Improved support for Intel+AMD hybrid graphics - just set it up by means of our XFdrake tool.
As for LXDE-specific improvements, one can mention only a couple of bug fixes:
- Fixed work of battery indicator on laptops.
- Fixed work of playing files with non-ASCII symbols in ROSA Media Player.
- Updating From Previous Releases
If you already have ROSA Fresh LXDE R4 installed then you have nothing to worry about - your system will be automatically updated to R5 state (and even further) by means of standard update mechanism. Users of older systems (R1, R2, R3) should first update their systems to R4 (though we recommend new installation wherever possible).
- Feedback
Feel free to try and provide your feedback. We have official forum monitored not only by isers but also by ROSA developers and QA team and classic bugzilla for advanced users who are able to formulate their problem in the terms that developers will easily understand. You can use this link to file a new bug against LXDE edition.
- Future Directions
The ROSA Desktop Fresh LXDE R5 is based on the rosa2014.1 with guaranteed support until the 2016, Autumn. However, before that date we are planning to switch to LXQt as our main lightweight desktop environment. LXQt appeared as a united project of LXDE and RazorQT teams who decided to rewrite all LXDE in Qt.
LXQt is already available in ROSA repositories and you can try it by installing task-lxqt package.
There are still a lot of things to do here, but the current quality is already close to LXDE
Basic GUI for ROSA Freeze
One of the new features in last ROSA Desktop Fresh releases was ROSA Freeze tool which allows you to "freeze" your system so that all (or almost all:)) modifications performed during the session are automatically rolled back after reboot. The tool is very easy to use but works from command line only. Many users are afraid of such tools and finally it is hard to find them in the system. To facilitate use of ROSA Freeze, we have developed a basic GUI application able to turn the freeze "on" and "off". The GUI tool doesn't have a special name, just look for the rosa-freeze-ui package. In command line one can use a shorter alias — urpmi rfreeze-ui. After this, you should be able to find "ROSA Freeze" in SimpleWelcome menu or just launch rfreeze-ui from the command line.
Besides turning the freeze on and off, GUI allows specifying different freeze parameters — what should be used to store modified versions of files and which folders should not be subjected to the freeze (remember that freeze of /home folder and folders located on a non-root partitions is not supported).
We hope that this simple GUI will make the use of ROSA Freeze easier. But be careful and don't forget that you have enabled the freeze (or don't be surprised if you loose all your changes after reboot). And if your system suddenly reports that no more space is left on your root partition (while you are sure that you have utilized only small part of it), check if you have the fereze turned on with tmpfs used as a storage for modified files.
As usual, you can find the source code of the new tool in ABF, suggestions, feedback and patches are welcome.
ROSA Cloud Connector - Easy Way To Connect The Cloud!
As some of you know, from time to time students from the Russian Higher School of Economics participate in ROSA development as a part of their studies. Usually the students perform different tasks concerning package maintenance and develop auxiliary tools (sometimes very useful, like this one).
Besides such simple tasks, the students perform more serious developments. Often we ask them to create some useful infrastructure programs which are not visible by ROSA users. However, this year we've got an exception - a tool named "ROSA Cloud Connector".
The tool behaves exactly like it name states and connects your machine to different cloud storage services. The connection can be performed only if the target service supports Linux. If there is a native client (like in case of Dropbox), it will be automatically installed. If there is no client but the service supports WebDAV, then this protocol will be used to mount your remote storage to the local system, so you will see your files in Dolphin, Nautilus and other file managers. You should only provide the program with your cloud service account details (login and password) and it will perform all necessary actions automatically - no need to download and install client or investigate WebDAV mounting instructions by yourself.
To get the tool in your system, you should install the rosa-cloud-connector package. Please make sure that all other packages in your system are up to date )especially those that contain "qt5" in their names). Then look for the rosa-cloud in the menu or just launch it from the command line.
The list of cloud storage services with Linux support is not huge. Moreover, it can become shorter over time - while we were working on the tool, two of the services known to us became dead. Though it is possible that we simply don't know that some other services exist - make us know if you are aware of such products and we will add their support to ROSA Cloud Connector. And it is possible that other Linux users will know new services after they meet them in our tool!
The current version of ROSA Cloud Connector is far from ideal (remember that it is the first Linux development experience for its authors). But if users express interest in it, we are ready to improve the tool.
As usual, everybody is welcome to provide not only feedback, but patches. The source code is available in ABF. Feel free to fork it, investigate and send pull requests!
ssd-utils - Weekly TRIM for SSDs
It is well-known that the low-level operation of solid-state drives (SSD) differs a lot from that of HDDs.
If the file system supports it, it can be beneficial to tell the SSD from time to time which blocks of data are no longer in use (deleted files, etc.). At least, this allows to avoid the performance degradation.
TRIM operation is provided exactly for that purpose and it can be used with the most of the modern SSDs. The file systems commonly used in Linux, like ext4, btrfs, xfs and some others, already support TRIM.
There are two main ways to get TRIM done:
- Perform TRIM each time a file is deleted. For example, this can be enabled for ext4 if the file system is mounted with discard option. Usually, it is not very convenient. TRIM takes time and, if it is done often, the file operations on this SSD may even become slower, negating the performance gain.
- Run fstrim command from time to time. TRIM will be performed for all data blocks that are no longer in use by the file system.
Ubuntu chose the second way starting from version 14.04. Now we have it in ROSA Fresh R4 too — just install ssd-utils package and that is it.
ssd-utils will perform fstrim right after the installation (for the file systems that support it) and will arrange for fstrim to run once a week automatically.
It is worth mentioning that the systems with encrypted volumes, RAID, devmapper-based volumes and the like may require some manual steps to make sure TRIM requests from the file system actually get to the underlying SSD.
Besides, similar to what Ubuntu does, fstrim will run only on the SSDs by the «reliable» vendors by default. At the moment, these vendors are:
- Intel
- Samsung
- OCZ
- SanDisk
- Patriot
There were problems with SSDs by some other vendors that caused file system corruption. Still, if you have an SSD by a vendor not listed above and believe the SSD is OK, you can enable fstrim for it too. To do that, just add --no-model-check option to fstrim-all in /etc/cron.weekly/fstrim.cron (see the comments in that file for details).
ROSA Desktop Fresh R4 LXDE
ROSA is happy to announce the release of LXDE edition of ROSA Desktop Fresh R4 distribution targeted to weak machines for which KDE or Gnome3 are too heavy. This release was highly requested by ROSA users and was prepared with the help of our community.
The distribution can be downloaded here:
Minimal system requirements:
- 256 Mb of RAM (recommended value is 512 Mb for the installed system, 384 Mb for the Live mode).
- Free hard drive space: 6 Gb
- CPU: Pentium4/Celeron
The core system components include:
- LTS kernel 3.14.15 with BFQ version 7r5
- Glibc 2.19
- GCC 4.9.2_2014.08 Linaro
- Graphical subsystem based on Xorg 1.15 and Mesa 10.2.7
- XFdrake tool for configuring graphical subsystem with improved support for hybrid Nvidia+Intel graphics.
- Latest versions of LXDE components based on thes Gtk stack
- Due to the lack of native tools for power management and notifications, appropriate components from XFCE are used which have been adapted to work with systemd.
The ISO image includes the following applications:
- LibreOffice Writer and Calc 4.3.2
- Firefox 32.0.2
- ROSA Media Player as a default media player
- Claws-mail mail client
Thus, for most users the distribution is ready to use out of the box. But even if some software is missing, users can install it from ROSA Desktop Fresh R4 repositories.
This is likely the last release of ROSA Desktop Fresh based on LXDE. In future we are going to use LXQt as a lightweight desktop environment. LXQt packages are already available in our repositories, so feel free to try them.
Urpmi - Automatically Add Media When Installing Remote Packages
ABF build system provides a convenient way to build experimental packages and share them with other users for test purposes without publishing them to any repositories — in order to do this, you can build a package and create a container from build list. Container is an ordinary repository which contains only packages from your build. So users can add your container as a repository and install packages from it.
However, containers are used as a temporary storage to tests packages before rejecting them or publishing to official repositories (and moreover, containers are automatically destroyed by ABF after two months from their creation). A typical scenario for users who would like to install packages from container is to add container as a repository, install packages and remove that repository. Too many actions! If you have only one package in container, you can simply provide urpmi with a link to that package, but this won’t work if you have several packages that depend on each other. No wonder that we got several requests to improve urpmi a little when dealing with remote files — urpmi should not only download a file, but try to add remote media (if possible) before installing the package, and remove this media after installation is finished
For example, if you want to install apache-mpm-prefork from build https://abf.io/build_lists/2290444. With previous urpmi, you couldn’t install it without adding container as a medium:
[root@r4null64 ~]# urpmi http://abf-downloads.rosalinux.ru/rosa2014.1/container/2290445/x86_64/main/release/apache-mpm-prefork-2.4.10-2-rosa2014.1.x86_64.rpm The following packages cannot be installed: apache-mpm-prefork-2.4.10-2-rosa2014.1.x86_64 (due to unsatisfied apache-base[== 2.4.10-2])
To be sure, apache-base-2.4.10-2 is located at the same container.
With the new urpmi, this command works like a charm:
urpmi http://abf-downloads.rosalinux.ru/rosa2014.1/container/2290445/x86_64/main/release/apache-mpm-prefork-2.4.10-2-rosa2014.1.x86_64.rpm
removing medium "medium_for_apache-mpm-prefork-2.4.10-2-rosa2014.1.x86_64.rpm"
adding medium "medium_for_apache-mpm-prefork-2.4.10-2-rosa2014.1.x86_64.rpm"
http://abf-downloads.rosalinux.ru/rosa2014.1/container/2290445/x86_64/main/release/media_info/20141009-184429-synthesis.hdlist.cz
http://abf-downloads.rosalinux.ru/rosa2014.1/container/2290445/x86_64/main/release/media_info/20141009-184429-info.xml.lzma
http://abf-downloads.rosalinux.ru/rosa2014.1/container/2290445/x86_64/main/release/media_info/20141009-184429-files.xml.lzma
http://abf-downloads.rosalinux.ru/rosa2014.1/container/2290445/x86_64/main/release/media_info/20141009-184429-changelog.xml.lzma
To satisfy dependencies, the following packages are going to be installed:
(test only, installation will not be actually done)
Package Version Release Dist DEpoch Arch
(medium "medium_for_apache-mpm-prefork-2.4.10-2-rosa2014.1.x86_64.rpm")
apache-base 2.4.10 2 rosa 2014.1 x86_64
apache-mod_actions 2.4.10 2 rosa 2014.1 x86_64
apache-mod_alias 2.4.10 2 rosa 2014.1 x86_64
< ... cut list of packages from container to be installed ... >
apache-mod_auth_basic 2.4.10 2 rosa 2014.1 x86_64
apache-mod_usertrack 2.4.10 2 rosa 2014.1 x86_64
apache-mod_version 2.4.10 2 rosa 2014.1 x86_64
apache-mod_vhost_alias 2.4.10 2 rosa 2014.1 x86_64
apache-modules 2.4.10 2 rosa 2014.1 x86_64
(command line)
apache-mpm-prefork 2.4.10 2 rosa 2014.1 x86_64
8.1KB of additional disk space will be used.
887KB of packages will be retrieved.
Proceed with the installation of the 38 packages? (Y/n) Y
< ... cut off installation log ... >
removing medium "medium_for_apache-mpm-prefork-2.4.10-2-rosa2014.1.x86_64.rpm"
As we can see, urpmi has added a medium named medium_for_apache-mpm-prefork-2.4.10-2-rosa2014.1.x86_64.rpm in the beginning and removed it in the end. Even if package installation fails (due to file conflicts, unresolved dependencies or if you answer «no» when urpmi suggests to install additional packages), the medium will be removed. But if you interrupt installation process (e.g., by pressing Ctrl-C), then medium will be left in the system.
URL of the medium to be added is obtained in a very straightforward way — we simply drop package name from the URL passed as urpmi argument. If urpmi fails to add such a media, you will get a corresponding message, but the installation of remote package will proceed. Finally, you can disable this behavior completely by using --no-auto-media option in command line or no-auto-media global option in /etc/urpmi/urpmi.cfg.
ROSA Hardware DB
The database is now available at Linux-Hardware.org.
Today the market offers a huge variety of configurations of personal computers. In the development of the ROSA operating system we make considerable efforts to support all possible configurations.
Five years ago, when developing first versions of the operating system and, until recently, we used the industry-standard method of user interaction. If the user encountered some problem, then he reported about it on our forum or bugzilla. The support team then began to ask the user to submit characteristics of the computer, system logs, etc. All these numerous data were collected in comments to the appropriate bug in bugzilla and then analyzed by the developers. The main disadvantage of this approach was that the user was required to perform too much actions, so the debugging of the problem stretched into weeks, and sometimes months.
To simplify the process of interaction with users, we have developed the HW Probe Tool. The tool is designed to collect all the necessary information on the user's computer to analyze and debug his problem. In this case, the user is required to perform only one command:
hw-probe -all -upload
One can run this command on the installed system or in the Live-mode. It is highly recommended to run this command as root in order to upload more details about all devices installed on the computer. And it's desirable to connect the maximum number of peripheral devices available to analyse them too.
As a result of this command, information about all hardware devices on the computer, the system initialization logs and other information will be uploaded to our database for subsequent analysis by developers and support team. User in this case will receive a link to the probe of his computer, that can be attached to the message on the forum, bug or shared with knowledgeable people who can help to fix the problem (the probe example for ASUS N73SV is here). As a result of this interaction mechanism, problems on users' computers are now analyzed and solved much faster.
The hw-probe package is already included in ROSA Desktop Fresh R4. Please update the package before using it in order to upload most complete test results to the database. Users of other versions of the ROSA operating system must install the package from this directory.
The database is available at hw.rosalinux.ru/.
On the basis of all the collected probes, as well as static analysis of the kernel drivers, we automatically create the database of supported hardware. You can find the DB on the website hw.rosalinux.ru/. You can, for example, look at the list of all tested graphics cards or the list of all WiFi-cards, support for which is declared in the kernel. You can also view the list and the classes of all tested PC models. For the classification of devices we use kernel classification of appropriate device drivers. For PCI and USB devices additionally we use thin classification by the class id of the device.
In this note, we invite all users of the ROSA operating system to upload probes of their computers in order to participate in creation of the ROSA Hardware DB. If some device does not work, please also post a message about it to our forum or bugzilla.
ROSA Desktop Fresh R4 Is Out!
The ROSA company is happy to present the long waited ROSA Desktop Fresh R4, the number 4 in the "R" lineup of the free ROSA distros with the KDE desktop as a main graphical environment. The distro presents a vast collection of games and emulators and the Steam platform package along with standard suite of audio and video communications software including the newest version of Skype. All modern video formats are supported. The distribution includes the fresh LibreOffice v4.3.1, the full TeX suite for true nerds as well along with best Linux desktop publishing, text editing and polygraphy WYSISYG software. The LAMP/C++/… enviroment packages are waiting to be installed by true hackers.
The R4 distribution is recommended for both the home users who want to communicate and surf the web and for the experienced ones who'd like to tweak the system their own way. ROSA Fresh R4 is based on a new rosa2014.1 platform which will be supported for 2 years and will serve as a base for several future releases.
Unlike R1, R2 and R3 releases, this release uses different repositories based on a new rosa2014.1 platform. All users of previous ROSA Desktop Fresh releases are highly encouraged to migrate to R4 release, since old releases are no longer supported. We recommend to install R4 over existing system from ISO image (if you have a separate /home partition, you can reuse it to save all your data), but it is also possible to update existing systems without re-installation.
Automated Metadata Updates in Urpmi
Maintainers who constantly deal with development branch of ROSA often meet a problem with outdated metadata when trying to install a package using urpmi - it often happens that a newer version of the pakcage has been already built and published, but the local metadata still contains only information about old version of the package which is now missing. If this is the case, one have to launch urpmi.update and try to install necessary package again. To be sure, one can simply launch urpmi --auto-update, but execution of this command can take significant time and usualyl maintainers don't want to wait for it.
A good news is that in rosa2014.1 development branch (which should become ROSA Desktop Fresh R4 soon) we have improved urpmi to automatiaclly update metadata and relaunch itself in case when something goes wrong with package download and it is likely that metadata update will help. This behavior is the default one in rosa2014.1, though one can disable it by means of «--no-restart» option. Note that to avoid looping, in any case the metadata will be updated only once - if this doesn't help, then urpmi will fail as before. In addition, you can specify a timeout for urpmi to wait until trying to update metadata. This option will be useful for ABF, since sometimes package builds fail due to the reason that old version of some package has been already dropped from repositories, while metadata for the new version hasn't been generated yet. When ABF will adopt a new urpmi, the number of such failures should decrease.
The new urpmi feature is primarily useful for maintainers. Most of ordinary users rarely meet such kind of problem, at least while they have automated updates enabled - the thing is that in order to perform automatic updates, urpmi.update is launch automatically on a regular basis, and updates in repositories of the released systems doesn't happen as often as in development branches. Though users are very different, so it is likely that some of them will notice that the number of routine actions necessary to update a package has decreased a little.
So, enjoy and be ready for the next ROSA release!
Spec-gen - Automated Spec File Generator for GNU Autotools and CMake-based Programs
Similar to many other distributions, ROSA pays a lot of attention to automation of different maintainers tasks and tries low the «entry barrier» for newbies. One of the problems faced by almost every newbie who wants to build some application for ROSA is creation of spec file to build an RPM package for our distribution. In many cases spec file can be generated automatically — for example, we have corresponding tools for modules of Python, Perl, Ruby, etc.
However, many programs for Linux are developed using comiled languages such as C/C++ and are built using GNU Autotools, CMake and analogues. Up to now, for such programs we only provided spec file templates and recommendations in Packaging Policies. This is normally enough for experienced maintainers, but newbies often experience difficulties with the very first step of spec file development. That’s why we have developed a new tool named spec-gen which is able to automatically generate spec file template on the basis of analysis of tarball with application source code.
The tool is very easy to use — just install spec-gen package in your ROSA installation and launch spec-gen with tarball name as an argument:
$ spec-gen my_tarball-1.0.tar.gz
By mean of -g, -s, -l and -u options you can specify group, summary, license and URL of the package to be created. You can omit these options and populate corresponding fields with data after the spec file is created. The tool try to guess package name and version on the basis of archive name — the guess is that most archives with source code are named in the «<program_name>-<version>» fashion. The tool supports all popular archive formats, but if you need to add support for more formats — feel free to send us a request.
During its work, the tool will look through archive content and check if it contains configure.in, configure.ac, CMakeLists.txt, *.cmake or setup.py files. If the latter file is found then the tool will simply launch python setup.py bdist_rpm5 command and provide your with a spec file created by it. In case when GNU Autotools or CMake files are detected, spec-gen will analyze them and detect which ROSA packages should be installed to build your application — that is, the tool will try to automatically create a list of Buildrequires for your new package.
We recommend to launch spec-gen in the system for which you want to build a package, since it uses information from system repositories about package provides and files by means of urpmq and urpmf programs.
As a result, spec-gen will provide your with a spec file with filled header, list of BuildRequires, %prep, %build and %install sections. Adjustment of the list of files in the %files section, splitting the package to subpackages and other complex tricks should be performed manually. To be sure, one can’t guarantee that generated BuildRequires list is completely correct — it is possible that some dependencies are not mentioned explicitly in build scripts or spec-gen makes a wrong choice among several alternatives. Nevertheless, in most cases the tool works correctly and provides you with a spec file that ca be used to build a package. It is likely that hte only task left to you is adjustment of the %files section.
It is possible that in future we will implement even more automation — for example, we can try to automatically build a package using generated spec file and then automatically adjust the %files section on the basis of build results. Actually there are a lot of possible improvements in this area, and everybody is welcome to provide su with feedback, share ideas and send patches. The source code is available here — https://abf.io/soft/spec-gen-dev.
Finally we are glad to note that the basis of the spec-gen tool was developed by Andrey Soloviev, a student of Higher School of Economics, during his practice in ROSA Company.
Boot and Install ROSA from your own HDD
There are many ways to install ROSA — for example, you can use ISO image located at your hard drive. However, a trick with ISO described in our wiki requires some full-functional operating system to be installed in your machine to perform some preliminary actions. However it can happen that your system is broken and neither you can boot from external device. Our colleague Sergey Sokolov recently met such a problem, and below he describes a solution that helped him to bring his system back to life.
Since I am one of ROSA developers, it is no wonder that I have installed a development release of ROSA Fresh R4 to my notebook (at the time when we even didn’t have an Alpha release). Once a day I decided to update my system to the current state of repositories. Unfortunately, it turned out that exactly at that moment a lot of system stuff updates (systemd, glibc, etc.) were ongoing. And I was so unlucky that my system refused to boot after update.
It would be nice to launch Live CD or just reinstall the system, but my machine didn’t have CD/DVD recorder, I didn’t have a boot USB flash and there was nobody near me to prepare such a boot device for me! However, I remembered that I had an ISO image of ROSA on my HDD.
I took a look at my partition table:
# fdisk -l /dev/sda Disk /dev/sda: 480.1 GB, 480103981056 bytes, 937703088 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sda1 2048 33556479 16777216 82 Linux swap / Solaris /dev/sda2 33556480 96471039 31457280 83 Linux /dev/sda3 96471040 937703087 420616024 5 Extended /dev/sda5 96473088 937703087 420615000 83 Linux
Here /dev/sda1 was a swap which I didn’t use anymore, /dev/sda2 was my root and /dev/sda5 was a /home.
The system failed to boot normally, but I still had initrd loaded and dracut console. And that turned out to be enough to do the following actions:
mkdir /mnt mount /dev/sad2 /mnt mount -o bind /dev /mnt/dev mount -o bind /dev/pts /mnt/dev/pts mount -o bind /proc /mnt/proc mount -o bind /sys /mnt/sys mount /dev/sda5 /mnt/home chroot /mnt dd if=/home/path/to/ROSA.FRESH.KDE.R3.x86_64.iso of=/dev/sda1 bs=8M touch /boot/resque.iso vi /boot/grub2/grub.cfg
In grub.cfg, I found rescue.iso item and edited it as follows:
### BEGIN /etc/grub.d/43_resque ###
if [ -f /boot/resque.iso -o -f /boot/sgb.iso ]; then
submenu 'Repair tools' {
if [ -f /boot/resque.iso ]; then
menuentry "Boot rescue CD" {
linux (hd0,1)/isolinux/vmlinuz0 boot=live iso_filename=/dev/sda1 root=live:/dev/sda1 rootfstype=auto ro rd.live.image rhgb splash=silent logo.nologo rd.luks=0 rd.md=0 rd.dm=0
initrd (hd0,1)/isolinux/initrd0.img
}
fi
Finally, I synced filesystem and rebooted. And now I had some kind of recovery partition which could be used to install a system or launch it in Live mode. Just not forget that we should not format sda1 partition when installing a system!
Rosabootstrap - setup a ROSA chroot in any Linux
It’s not a secret that some developers of ROSA and OpenMandriva work on our distributions every time they have a small chance to do this. And it is not surprising that sometimes they just don’t have a ROSA/OpenMandriva system under the hand when they have a half an hour to spend it to development. Downloading ISO image and setting it up is not a good solution, since it requires some time and machine resources.
However, now they can setup a minimalistic chroot with ROSA or OpenMandriva in any Linux distribution which provides Shell and Python (even rpm is not necessary!). Thanks to Robert Xu, now we have omvbootstrap and rosabootstrap tools that allow doing this easily.
In case of ROSA, just clone the rosabootstrap project:
$ git clone https://abf.io/soft/rosabootstrap $ cd rosabootstrap
or download and unpack the tarball:
$ wget https://abf.io/soft/rosabootstrap/archive/rosabootstrap-master.tar.gz $ tar xzvf rosabootstrap-master.tar.gz $ cd rosabootstrap-master
And launch the script with necessary parameters:
$ sudo ./rosabootstrap -d -a x86_64 -v 2012.1 -c 2012.1 -m http://mirror.yandex.ru/rosa/rosa2012.1/repository/x86_64/main/release
That’s all — now you should have 2012.1 folder with chroot, so just go to it and enjoy the coding:
$ sudo chroot 2012.1
Usage of our developer tools in upstream
During the development of operating systems, our developers and maintainers use a huge variety of program tools (in order to build packages, analyze code quality, verify changes in code, etc.). Most of the tools are available in almost any repository, such as, for example, rpmbuild, gcc, rpmlint, check, valgrind, diff, etc. But sometimes there are tasks for which tools have not yet been created. In such cases, we create our own tools for developers. If these tools can be helpful not only for us but also for the community, then we publish their source code.
An example of one of the non-standard tasks was to analyze the backward API/ABI compatibility of system libraries in our operating system ROSA. Because the number of libraries in the system reaches several thousands, the manual tracking of changes was too difficult task. Therefore, we have developed the ABICC tool, that can automatically analyze the compatibility of changes in libraries. The source code of this tool have been published and gradually more and more upstream developers use this tool to control API/ABI compatibility of interfaces in their libraries. As a result, our maintainers can easier update appropriate packages for such libraries.
Examples of libraries successfully using our tools are: Pacemaker, MySQL++, Wireshark, Glibc, Enlightenment, libDAP++, libapt, Barry, PySide, PLplot, etc. Also, quite a number of library developers prefer to use our special service Upstream Tracker, where they can be free add any library and monitor changes in its API/ABI interface. Examples of such libraries are ImageMagick, V8, etc.
Our most popular open-source tools for developers can be found here. Among them are the following tools:
- ABICC
- a tool to check backward API/ABI compatibility of system libraries.
- PkgDiff
- a tool to classify files and visualize changes in source packages.
- Java ACC
- analogue of ABICC for Java libraries.
- API Sanity Checker
- an automatic generator of automatic unit tests for C/C++ libraries.
- ABI Dumper
- extract ABI structure from the debug-info of a shared library.
- Vtable Dumper
- list content of virtual tables in a shared library.
We encourage upstream developers to use our tools. This can improve the quality and stability of their API/ABI interfaces. And moreover this simplifies updating of appropriate packages in operating systems.
ROSA Desktop Fresh R3 KDE
Good news, everyone!
ROSA Desktop Fresh R3 release is out with KDE desktop environment.
ROSA Desktop Fresh R3 is a regular update of the ROSA Fresh platform. The R series of distributions is targeted to those users who looks for fresh and full-functional software. This series is developed by ROSA with significant help of community. The release includes all updates, fixes and improvements implemented since the time the previous R2 version was released.
Users who already work in ROSA Desktop Fresh R2 are able to update their systems to ROSA Desktop Fresh R3 from official repositories.
Newbies can easily download and try it, without reading the notes below.
GUI for urpmi.recover
Last week we introduced urpmi.recover - a tool able to rollback the state of your system's package base. The new tool is already used actively and we have got a lot of useful feedback and fixed some issues.
One of the main concerns with urpmi.recover is that it is not very convenient to select a date or number of transactions to be reverted using command-line interface. To solve this problem, we quickly implemented a simple GUI for urpmi.recover; to be more precise, we have extended qt4urpm to support the new tool.
Qt4urpm is a small program developed several years ago by members of MandrivaUser.de community to perform two operations with urpmi repositories - file search and orphan detection and removal. To be sure, such a functionality is partially provided by Rpmdrake, but qt4urpm is much more lightweight (it is written in Qt) and faster. Despite its simple interface, it is sometimes is even more functional than Rpmdrake - for example, by means of qt4urpm you can remove only selected orphan packages. And when working with qt4urpm, you will rarely need to perform more than three mouse clicks.
So we have decided that a simple and straighforward GUI for urpmi.recover will perfectly fit in qt4urpm, and it didn't take much time to implement this. Install the freshest version (2.0) of qt4urpm, launch the program and select the urpmi.recover tab. Here you will see a list of transactions sorted by date. For every transaction, a set of packages is displayed which were installed during it:
Now select any transaction and press Rollback - urpmi.recover will revert all transactions starting from the selected one. That's all, folks:)
Maybe in future our GUI experts will implement something more beautiful, but from functionality point of view, qt4urpm already provides all the information you need to play with you package base state. We hope that it will help you to avoid getting lost in rpm transactions and to feel more comfortable when rolling back your system to some date in the past.
Spec-cleaner and rediff patch - New Helper Scripts for Maintainers
Maintainers' work includes a lot of routine tasks — adoption of spec files to match evolving packaging policies and build tools, adoption of existing patches for new upstream versions, update of package file lists and so on. Some of such tasks are performed automatically by build tools — rpmbuild itself and auxilliary scripts from spec-helper package. However, these scripts by design can’t solve a lot of problems — in particular those that require modification of spec files.
To simplify maintainers' life a little more, we have included two new scripts to the spec-helper package. The scritps' names are self-explaining — spec-cleaner and rediff_patch. These scripts are not launched automatically during the build; they are intended to be used by maintainers manually. Update spec-helper package in your system and you will get these scripts in your /usr/bin.
Spec-cleaner performs the following actions:
- removes obsolete declarations that makes no sense nowadays — for example, definitions of BuildRoot and Packager, requirement on install-info, buildroot cleanup and so on;
- modifies formatting of macros and variables — according to our policies, variables in spec files should be emraced with braces — %{const}, while for macros braces should not be used — %{macro}. Note that spec-cleaner only reformats macros and variables known to it. If you have some entities defined by your own, the script will leave them «as is»;
- formats Summary — capitalize the first letter and removes the dot from its end;
- removes explicit declarations of %{name}, %{version} and %{release} variables;
- replace obsolete macros with modern analogues;
- … and performs a lot of other small tasks which will help to make spec files smaller and cleaner and avoid claiming from rpmlint.
When developing spec-cleaner, we try to avoid situations when the new spec-file becomes incorrect in some sense. Due to this reason, some spec-file issues that may seem to be easy to fix are not automatically fixed yet.
Meanwhile, old developers may notice that spec-cleaner is a modern and much powerful analogue of old macroszification script which was present in spec-helper for years. Now this script is dropped — use new spec-cleaner instead.
As for Rediff_patch, then you have likely guessed that it tries to rediff an existing patch against a new tarball with source code. Please read the instructions below carefully before trying to use this script:
- rediff_patch should be launched in the folder of a cloned Git project, where spec file and patches reside. Spec file is used to determine how the patch should be applied;
- the new tarball should be placed in the same folder;
- the script should be launched in the following way:
rediff_patch <patch_ro_rediff> <tarball>
If you folder contains only one tarball, then you can omit second argument.
During its work, rediff_patch creates rediff_patch folder, unpacks tarball into it and tries to apply the patch to the unpacked content. To apply the patch, it uses parameters extracted from the spec file, but adds «--force» option and uses default system value for the «fuzz» option (remember that rpmbuild uses «--fuzz=0» by default). Currently the script only handles situation when tarball has a single top-lever folder; it will reject to work with tar bombs
If everything goes fine, then you will find a new patch (with the «.new» suffix) near the old one. rediff_patch folder with the whole content will be removed.
If something goes wrong (e.g., patch was applied only partially and some hunks failed) then you will be left with rediff_patch folder containing two subfolders — the original one and the new one, where a patch was tried to be applied So you will be able to analyze the reasons of failures and finish rediff manually.
As our practice shows, in reality most patches fail to rediff completely automatically, even with «--force» and more soft «--fuzz». But even rediff_patch managed to prepare a new patch for you, do not forget to check this patch carefully — '--force' sometimes leads to undesired results. But even if rediff_patch failed — well, at least we have saved some time, since it has unpacked tarball for us and performed a first attempt to apply the patch.
We hope that these two new scripts will make your life a little easier. Both scripts are easy to use and straightforward. As usual, comments, suggestions and patches are welcome.
Urpmi.recover - "Back In Time" For The Package Base
Many developers and advanced users often meet a need to rollback recently installed packages that introduced some undesirable updates to their system. this usually happens when you install packages from third party or testing repositories or from private repositories and containers where maintainers perform test builds of their packages. The latter case is quite common for ROSA QA team whose members often meet a need to rollback a set of packages installed for testing purposes.
Manual rollback is not very convenient, especially if the number of packages is huge and you are not completely sure which of them should be erased or downgraded to return system to a full-functional state. To be sure, in many cases all problems can be solved by urpm-reposync, but sometimes this tool is too powerful — it performs complete synchronization of your system and enabled repositories and it’s not easy to rollback only several packages.
A good news that now a niche between manual rollback and reposync usage is filled by urpmi.recover tool which is able to rollback your package updates. Urpmi.recover will help you to rollback your package base to certain date in the past or just rollback a given number of transactions.
Urpmi.recover is included in urpmi package and will be installed to your system with regular updates.
To be able to perform such a rollback, urpmi.recover stores old versions of updated packages in /var/spool/repackage folder. To start use the tool, you should first initialize the backup of old package versions by typing
# urpmi.recover --checkpoint
By typing this command, you say: «Currently my system is in stable state, but I am going to install dangerous updates. Please, starting from this moment, track all newly installed packages and backup the old versions in case of updates».
You can also run this command at any moment in future to rebase the stable system state. Every invocation of urpmi.recover --checkpoint will clean the /var/spool/repackage folder, so you won’t be able to rollback to earlier dates.
While package updates tracking is enabled, old versions of updated packages are stored in /var/spool/repackage subfolders corresponding to update date, so you can always rewise these old versions by yourself.
At some moment when you decide that it is time to revert your packages (at least to try to do it:)), simply say something like:
# urpmi.recover --rollback <timestamp>
You can specify timestamp in «seconds since the Unix Epoch», but feel free to use human-readable formats, e.g.:
# urpmi.recover --rollback "2014-03-07 13:20:47"
or even
# urpmi.recover --rollback "1 hour ago"
You can also rollback a given number of transactions by specifying --transactions option and passing number of transactions to be reverted to --rollback option:
# urpmi.recover --transactions --rollback <number_of_transactions>
For example, if you just updated a package a want to rollback this update, you can tell urpmi.recover to revert a single transaction:
# urpmi.recover --transactions --rollback 1
Finally, to completely disable repackaging and to clean /var/spool/repackage folder, just type:
# urpmi.recover --disable
This command will also clean up the /var/spool/repackage folder.
So these are the ways you can use urpmi.recover to rollback your package base. The tool is in experimental stage; we don’t guarantee absence of aerrors, so use it on your risk. though it should be said that before performing the rollback, urpmi.recover will provide you with details — which packages it is going to drop and which ones to downgrade, and you will be asked for confirmation. Finally, you can always use urpm-reposync in case of emergency.
One should also remember that maintainers usually don’t care about possibility to correctly downgrade their packages. So if new version of a package introduced some problems to your system, it is not necessary that rollback to a previous version will help to bring your system to a functional state.
Multiboot flash drive with several ROSA editions
Want to have a single flash drive able to boot several editions of ROSA? Sergey Zhemoitel provides a set of instructions on creating a multi-boot flash drive with several ROSA editions using grub4dos.
As an example, we will setup a flash able to boot 32bit and 64bit images of ROSA Desktop Fresh KDE. We assume that /dev/sdX is a device corresponding to your flash drive.
- Install grldr.mbr to flash MBR:
dd_rescue grldr.mbr /dev/sdX
- Create two partitions using fdisk or diskdrake:
- /dev/sdX1 - 200 Mb
- /dev/sdX2 - the rest of the device
- Format partitions:
- /dev/sdX1 - ext2 (grub4dos)
- /dev/sdX2 - ext4
- Copy grldr and menu.lst to /dev/sdX1
- In /dev/sdX2, create two folders for our ISO images:
mkdir -p rosa/kde/x86_64 rosa/kde/i586
- Unpack ISO images to corresponding folders
- Edit menu.lst and reboot.
menu.lst should look like the following:
default /default title ***** ROSA Linux KDE R2 x86_64 ****** root title ROSA install find --set-root --ignore-floppies /rosa/kde/x86_64/isolinux/isolinux.bin kernel /rosa/kde/x86_64/isolinux/vmlinuz0 root=live:UUID=40af22cf-3bab-48f4-841b-9d4fffdd87df rootfstype=auto ro rd.live.image live_dir=/rosa/kde/x86_64/LiveOS rhgb splash=silent logo.nologo install vga=788 initrd /rosa/kde/x86_64/isolinux/initrd0.img title ROSA Live find --set-root --ignore-floppies /rosa/kde/x86_64/isolinux/isolinux.bin kernel /rosa/kde/x86_64/isolinux/vmlinuz0 root=live:UUID=40af22cf-3bab-48f4-841b-9d4fffdd87df rootfstype=auto ro rd.live.image live_dir=/rosa/kde/x86_64/LiveOS vga=788 desktop nopat rd.luks=0 rd.lvm=0 rd.md=0 rd.dm=0 noiswmd splash=silent quiet logo.nologo initrd /rosa/kde/x86_64/isolinux/initrd0.img title Verify and Boot ROSA.Desktop.Fresh.R2.2012.x86_64 find --set-root --ignore-floppies /rosa/kde/x86_64/isolinux/isolinux.bin kernel /rosa/kde/x86_64/isolinux/vmlinuz0 root=live:UUID=40af22cf-3bab-48f4-841b-9d4fffdd87df rootfstype=auto ro rd.live.image live_dir=/rosa/kde/x86_64/LiveOS rhgb vga=788 splash=silent logo.nologo rd.live.check initrd /rosa/kde/x86_64/isolinux/initrd0.img title Install ROSA Desktop.Fresh R2 2012 in basic graphics mode. find --set-root --ignore-floppies /rosa/kde/x86_64/isolinux/isolinux.bin kernel /rosa/kde/x86_64/isolinux/vmlinuz0 root=live:UUID=40af22cf-3bab-48f4-841b-9d4fffdd87df rootfstype=auto ro rd.live.image live_dir=/rosa/kde/x86_64/LiveOS rhgb vga=788 splash=silent logo.nologo install xdriver=vesa nokmsboot install initrd /rosa/kde/x86_64/isolinux/initrd0.img title Rescue ROSA Fresh R2 2012 x86_64 find --set-root --ignore-floppies /rosa/kde/x86_64/isolinux/isolinux.bin kernel /rosa/kde/x86_64/isolinux/memdisk initrd /rosa/kde/x86_64/isolinux/sgb.iso title ***** ROSA Linux KDE R2 i586 ***** root title ROSA install find --set-root --ignore-floppies /rosa/kde/i586/isolinux/isolinux.bin kernel /rosa/kde/i586/isolinux/vmlinuz0 root=live:UUID=40af22cf-3bab-48f4-841b-9d4fffdd87df rootfstype=auto ro rd.live.image live_dir=/rosa/kde/i586/LiveOS rhgb splash=silent logo.nologo install vga=788 initrd /rosa/kde/i586/isolinux/initrd0.img title ROSA Live find --set-root --ignore-floppies /rosa/kde/i586/isolinux/isolinux.bin kernel /rosa/kde/i586/isolinux/vmlinuz0 root=live:UUID=40af22cf-3bab-48f4-841b-9d4fffdd87df rootfstype=auto ro rd.live.image live_dir=/rosa/kde/i586/LiveOS vga=788 desktop nopat rd.luks=0 rd.lvm=0 rd.md=0 rd.dm=0 noiswmd splash=silent quiet logo.nologo initrd /rosa/kde/i586/isolinux/initrd0.img
Note: the following parameters are obligatory:
- root=live:UUID=40af22cf-3bab-48f4-841b-9d4fffdd87df
- live_dir=/rosa/kde/i586/LiveOS
the former specifies a partition where the unpacked image resides, the letter points to a folder with squashfs.img file.
UUID can be obtained using blkid command:
# blkid <...> /dev/sdc1: LABEL="grub4dos" UUID="74e94dfa-6b1d-48ec-96bd-d96c66e55400" TYPE="ext2" /dev/sdc5: LABEL="flash" UUID="40af22cf-3bab-48f4-841b-9d4fffdd87df" TYPE="ext4" /dev/sdc6: UUID="2013-11-29-20-39-56-00" LABEL="ROSA.FRESH.KDE.R2.i586" TYPE="iso9660" PTTYPE="dos" /dev/sdc7: UUID="2013-11-29-17-24-42-00" LABEL="ROSA.FRESH.KDE.R2.x86_64" TYPE="iso9660" PTTYPE="dos"
Here we can see UUID of our partition with ISO images - "40af22cf-3bab-48f4-841b-9d4fffdd87df".
