Friday, November 14, 2008

kernel c++

C++ for Kernel Mode Drivers: Pros and Cons

Updated: February 1, 2007
On This Page
Introduction Introduction
Considerations for Kernel-Mode Code Considerations for Kernel-Mode Code
Using the C++ Compiler for Kernel-Mode Code Using the C++ Compiler for Kernel-Mode Code
C++ Issues for Kernel-Mode Drivers C++ Issues for Kernel-Mode Drivers
Summary Summary
References References

C++ with its object features appears to be a natural match for the semantics of Microsoft Windows Driver Model (WDM) and Windows Driver Foundation (WDF) drivers. However, some C++ language features can cause problems for kernel-mode drivers that can be difficult to find and solve. To help you make an informed choice, this paper shares current insights and recommendations from Microsoft's ongoing investigation of using C++ to write kernel-mode drivers for the Windows family of operating systems.

This information applies for the following operating systems:
Microsoft Windows 2000
Microsoft Windows XP
Microsoft Windows Server 2003
Microsoft Windows Vista
Microsoft Windows Server 2008

Introduction

With its object features, C++ appears to be a natural match for the semantics of Microsoft Windows Driver Model (WDM) and Windows Driver Foundation (WDF) drivers, and it is appealing for the added convenience and expressive power it provides to developers. However, technical issues associated with writing kernel-mode code in C++ using the currently available Microsoft compilers can cause problems in driver code.

Many developers use the C++ compiler as a "super-C" without using the full C++ language, because the C++ compiler enforces certain rules more strictly than standard C compilers and provides some additional features that happen to be safe for use in the context of drivers. Using the C++ compiler in this way is typically expected to work for kernel-mode code. It is "advanced" C++ features such as non-POD ("plain ol' data", as defined by the C++ standard) classes and inheritance, templates, and exceptions that present problems for kernel-mode code. These problems are due more to the C++ implementation and the kernel environment than to the inherent properties of the C++ language.

Microsoft is investigating issues related to using C++ to write kernel-mode drivers for the Microsoft Windows family of operating systems. This paper shares current insights from Microsoft developers about the tradeoffs of writing drivers in C++.

The information in this paper applies to the standard Windows Driver Development Kit (DDK) build environment for creating kernel-mode drivers as of the Windows Server 2003 Service Pack 1 (SP1) DDK. If you are using build environments or compilers other than those provided with the DDK or Windows Driver Kit (WDK), you should determine whether any of the issues noted here apply to your development environment and whether there are additional concerns. The information to determine this might be available as documentation from the compiler provider, but it is more likely that you will have to inspect generated code and link maps, as discussed below.

This paper does not attempt to explain how to write kernel-mode drivers in C++. This paper assumes that you understand the basic principles of writing kernel-mode drivers. For general information about writing kernel-mode drivers, see the Kernel-Mode Architecture Guide and device-specific information in the Windows DDK documentation.

Considerations for Kernel-Mode Code

Kernel-mode code must take into account the following considerations, to avoid data corruption, unstable systems, and operating system crashes.

The kernel manages its own memory pages:
You must manage the two conflicting requirements of correct operation and minimizing memory footprint.

•

Code and data must be in memory if code is to be executed when paging is not allowed. That is, when the system is running at IRQL DISPATCH_LEVEL or higher, the pages that contain the currently executing routine, any routines that it calls, or data it accesses (and so on down the chain of function calls) must be locked into memory until the IRQL drops below DISPATCH_LEVEL. Otherwise, a page fault occurs and the system crashes.

•

To increase the amount of memory available for user applications, drivers should make their code and data segments pageable where reasonable. This can improve system performance.

Not all processor resources are available all the time.

•

On x86 systems, the floating point and multimedia units are not available in kernel mode unless specifically requested. Trying to use them improperly may or may not cause a floating-point fault at raised IRQL (which will crash the system), but it could cause silent data corruption in random processes. Improper use can also cause data corruption in other processes; such problems are often difficult to debug.

•

On Intel Itanium systems, not all of the floating-point registers are available.

Resources, particularly the stack, are severely constrained. Resources that are "inexpensive" in user space may be expensive or require different methods to obtain in kernel mode. Specifically, the size of the kernel stack is 3 pages.

Not all of the standard libraries (C or C++) are present in kernel mode.

•

Versions of standard libraries provided with the build environment for use in kernel mode are not necessarily the same as those provided in user mode, because they cannot rely on the Win32 API and because they must be written to conform to kernel mode requirements. Kernel-mode implementations of standard libraries may have limited functionality or be constrained by other properties of kernel mode.

•

User-mode implementations of library routines might not work in kernel mode. Some do not link, some do not run, and some might appear to run, but with unintended side-effects.

Using the C++ Compiler for Kernel-Mode Code

It is important to remember that the compiler generates correct object code, but it may not be the code you expect, organized in the way that you expect. This is always true, but it is more likely to be a problem for C++ than for C. You must examine the object code to be sure it matches your expectations, or at least will work correctly in the kernel environment.

Output from the currently available C++ compiler is not guaranteed to work in kernel mode across all platforms and versions. The more your code uses the "advanced" C++ features, the more you risk problems with interoperability.

Key Areas for Kernel-Mode Code
The following areas require particular care in kernel-mode drivers. These apply to both languages (C and C++), but may be more problematic in C++ code because the compiler does more things automatically, and you may not realize that it has created a problem.

•

Floating-point instructions must be properly protected—for example, with KeSaveFloatingPointState and KeRestoreFloatingPointState or other mechanisms described in Windows DDK documentation.

•

The InterlockedXxx functions should insert memory barrier instructions in the generated code. Check the output to ensure that the barriers you require are present.

•

The semantics of the volatile keyword must be carefully understood so that the intended level of indirection is the volatile object. Sometimes the volatile item is the pointer, sometimes it is the object itself, and sometimes both pointer and object are volatile. Applying the volatile keyword to the wrong thing is a common error, so carefully check your use of this keyword. For example, if you intend to use a non-volatile pointer to a volatile location, ensure (by careful code reading) that your code does not implement a volatile pointer to a non-volatile location.

•

Stack frames are severely limited. For example, on x86 systems the total stack available to a thread is 12K.

•

Non-obvious jumps or memory usage in function source code creates the risk of an unexpected page fault. Specifically, the compiler can generate functions and data objects whose existence is not immediately obvious. For details about objects that might not be expected, see "Code in Memory" later in this paper.

•

The use of inline functions (and __forceinline), to ensure that the code is resident in memory interacts with the compiler's optimization rules.

•

A function you expect to be inlined might not be inlined. Consequently, using the function might cause a page fault.

•

The compiler might generate inline code for a function when you do not expect it to.

Safe and Unsafe C++ Constructs
Although it is not currently possible to provide a strict and testable definition of the "completely safe" subset of C++ for use in kernel-mode code, some useful guidelines are available for constructs that are usually safe and those that are usually not.

A good rule of thumb is that a C++ construct is probably safe if there is an obvious way to rearrange the code to make it legal C. An example is the relaxed ordering of declarations, including declaring variables in for statements.

The stricter type checking in C++ may disallow a technically legal, but semantically incorrect, construct. Such stricter type checking is a useful means of improving the reliability of the driver.

Anything involving class hierarchies or templates, exceptions, or any form of dynamic typing is likely to be unsafe. Using these constructs requires extremely careful analysis of the generated object code. Limiting use of classes to POD classes significantly reduces the risks.

Reviewing Generated Code
One of the original design goals for C was that it be fairly easy to determine what the generated object code would be, thus making it quite suitable for kernel-mode work. C++ is a much more complex language, and consequently making it work in the kernel environment has proven to be much more difficult.

To write drivers in C++, you must understand the code generated by the compiler, ensure that it meets kernel-mode requirements, and ensure that it avoids the problems discussed in this paper. Be prepared to read the object code and to scan the link map to be sure that code and data are placed in the proper locations and that only kernel-safe libraries are used. Check code for pageability, inline functions, and correct program ordering.

We strongly recommend that you begin such code reading and testing now, rather than waiting until the source code is complete. Check early prototypes and test potentially troublesome usage so that if you encounter an insurmountable problem with C++, you have time to find and implement an alternative solution.

C++ Issues for Kernel-Mode Drivers

Microsoft developers have discovered a number of areas where C++ presents particular problems for kernel-mode drivers.

Code in Memory
The most severe problem with using C++ for writing kernel-mode drivers is the management of memory pages, particularly code in memory, rather than data. It is important that large drivers be pageable, and paged code is not always in memory. All of the code that will be needed must be resident before the system enters a state in which paging cannot occur.

The way the C++ compiler generates code for non-POD classes and templates makes it particularly difficult to know where all the code required to execute a function might go and thus difficult to make the code safely pageable. The compiler automatically generates code for at least the following objects. These objects are put "out of line," and the developer has no direct control over the section in which they are inserted, which means they could happen to be paged out when needed.

•

Compiler-generated code such as constructors, destructors, casts, and assignment operators. (These can often be explicitly provided, but it requires taking care to recognize that they need to be provided.)

•

Adjustor thunks, used to convert between various classes in a hierarchy.

•

Virtual function thunks, used to implement calls to virtual function.

•

Virtual function table thunks, used to manage base classes and polymorphism.

•

Template code bodies, which are emitted at first use unless explicitly instantiated.

•

The virtual function tables themselves.

The C++ compiler does not provide mechanisms for direct control of where these entities are placed in memory. The pragmas necessary to control memory placement were not designed with C++ in mind. #pragma alloc_text cannot be used to control the location of a member function because (for several reasons) there is no way to name the member function. The scope of #pragma code_seg is ambiguous for compiler-generated functions, expanded template bodies, and compiler-generated thunks. There is no mechanism at all for controlling the location of virtual function tables, since they are not quite either code or data from the point of view of the compiler (they go into a section all their own).

If a function in a header is declared inline, but the compiler does not generate inline code for it, the function may be emitted in more than one code segment depending on where the function is used. When a class template is instantiated, it is generated in the section that is current at the point of first use, and it is not always immediately obvious which section that is. Both of these issues can lead to code being pageable when it should not be, or vice versa.

If a class hierarchy is in use, whether code for a base class needs to be in memory when the derived class is accessed depends on exactly which functions in the base class are called from the derived class (and whether the compiler can inline them), as well as what sections they were emitted in. For example, if the derived class provides a method that uses no base class methods, the base class code need not be in memory. However, it is difficult to know when that is the case. Additionally, any thunks used with the hierarchy and its classes might also need to be resident in memory.

Stack
The compiler has always been free to generate additional data on the stack, such as creating temporary objects, deferring call cleanup, and other actions that use the stack in a hidden fashion. There are few differences between C and C++ with respect to the way a single function uses the stack, but because of the additional mechanisms that usually result in more function calls, C++ will often use more total stack. You should keep stack size in mind, as you would in any programming language when stack space is limited.

Exceptions also have an effect on the stack. See "Exceptions and RTTI" later in this paper.

Dynamic Memory
Driver development tools such as Driver Verifier rely on tagged memory to validate memory usage in drivers. Using operator new and operator delete to allocate and free memory weakens the ability of these tools to detect memory leaks and other problems in driver code.

In user space, operator new and operator delete are convenient, but they can become cumbersome in drivers that use multiple memory pools or tagged memory. Because "placement new" takes additional operands, it is possible to pass in the information needed to select memory pools or generate tags into an overloaded operator new , but this is not much easier than using the memory functions directly. Because there is no "placement delete" with additional arguments to pass in a tag or a pooltype, there is no way to pass in a tag (or memory control, if needed) when using operatordelete, making it impossible to check that the tag at the point of release was the intended one, thus defeating much of the benefit of using tagged memory. It is possible to delete memory without providing a tag, but in each case you will need to decide whether the risks and disadvantages of not using tags in driver code overcome the apparent convenience.

Memory tracing tools often record the return address of the function that made an allocation. Some C++ compilers implement operator new as a function, causing all allocations to appear to come from a single location and defeating the purpose of that aspect of the memory tracing tool. This can be addressed, but you will have to determine for yourself if there is a benefit in doing so over using memory allocation directly.

Libraries
There are a number of distinct concerns in creating and using libraries:

•

The name of exported C++ functions can vary from one release to another.

•

Not all of the functions available in user mode are available in the kernel-mode libraries.

•

The Standard Template Library is designed to work with data objects from a single DLL.

C++ functions are exported based on their entire signature, not on their name alone (as C functions are). The name of a C++ function is "mangled" to contain type information, which becomes part of its signature. Although the rules for name mangling are fairly stable, there is no guarantee that the mangled names will be the same from release to release of the compiler. Therefore, C++ functions cannot be reliably exported to a library from one release to the next, although functions that can be represented as extern "C" functions can. In addition, the use of a .def file can help mitigate the problem. Note that extern "C" functions are unique only on the basis of name, not the entire signature as in C++.

Not all library functions are available in kernel mode, particularly those associated with the "advanced" C++ language features. The Standard Template Library is the "usual" way to implement many C++ concepts such as variably sized arrays. However, it is unsafe to simply assume that the Standard Template Library is present and usable. Although much of the Standard Template Library is implemented as source code in headers, it occasionally uses library functions or other features that are not available or usable in the kernel environment.

The Standard Template Library is also based on the assumption that each data object it uses exists in only a single DLL. Although in most cases it works to pass references to POD objects across DLL boundaries, passing references to more complex structures such as lists may cause runtime failures that can be hard to diagnose. Known issues include the fact that freeing memory in a DLL other than the one in which the memory was allocated can cause failures (at least for debug-mode compiles) and that the "end of list" marker differs between DLLs, which can cause unexpected runaway list searches. You must be aware of these problems and take steps to prevent them.

We do not recommend using Standard Template Library functions in a kernel-mode driver, because it is not possible to assume that the Standard Template Library is there and "just works." In the case of kernel-mode code, understanding precisely how a particular data structure is implemented helps assure that it does not violate the requirements of kernel space. It is also possible that a specialized implementation will be smaller than the more general Standard Template Library functions, although the library is often very good in that regard.

Exceptions and RTTI
It is tempting to use C++ exceptions, but they are difficult to implement in kernel mode. C++ exceptions require a kernel-mode-safe library, which does not currently exist. They also present an unavoidable runtime problem, because exception records that are generated when an exception is thrown are large objects on the very limited stack. On x86 systems, exception records are not particularly large (although they are large compared with many typical stack frames), but on Intel Itanium systems they are quite large: 3K to 4K, or one-sixth to one-eighth of the available 24K stack space. To preserve portability of a driver to 64-bit platforms, exceptions would have to be used in a very limited way, even on the x86 architecture. The rethrow operator can cause multiple exception records on the stack. Note that Structured Exception Handling (__try /__except/__finally) is available in kernel mode, although the space concerns remain. C++ exceptions have a number of semantic subtleties that prevent them from simply mapping onto Structured Exception Handling.

Run-time type information (RTTI) also requires a library that does not currently exist for C++ in kernel mode. So far, there have been few, if any, requests for this in kernel-mode code. Whether this lack of demand is a consequence of the other problems masking it or because it is not useful in kernel mode is unknown.

Compiler Versions
Although the C++ language standard is stable, implementation techniques are still evolving. Consequently, compiler versions may change the way generated code operates. Such changes are unlikely to affect user-mode code, but they can affect kernel-mode code in which more of the underlying implementation is exposed to (and sometimes provided by) driver developers; version-to-version interoperability of kernel-mode code is not guaranteed.

You should carefully control any interface between two drivers or a driver and the operating system, usually by writing the interface in C instead of C++. Otherwise, version-to-version incompatibilities in the C++ implementation may cause interoperability failures.

Static and Global Scope Variables and Initialization
C++ static variables (declared at either global or local scope) present a number of problems for drivers.

The C++ standard allows static variables declared at local scope to be initialized at the time of first use (the first time the scope is entered). The way this is implemented both creates the possibility of race conditions during initialization and a particularly high risk of unintended data sharing between threads, because variables declared static are globally static, not per-thread. For globally static data (shared among threads) it is best to do it explicitly at global scope, to make sure access protections appropriate to the situation are applied.

If a C++ global object requires initialization (a global constructor) is declared, there is no mechanism for the constructor to be called. Global objects that require constructors should either not be used, or you must develop a mechanism to assure that the constructor is called. Several sources on the Web claim to have solved this problem, and one of those solutions might work for you.

The order of initialization of global objects is not specified by the C++ standard, so even if there were a mechanism to call their constructors, either the order of initialization must be explicitly controlled by the driver code, or it must not matter.

Summary

Microsoft neither endorses nor prohibits the use of C++ for kernel-mode drivers. This conservative position is driven in part by the issues described in this paper, and in part by the need to support all platforms. You must be aware of known problems and risks described in this paper before attempting any development in C++ for kernel mode, and you should be alert for other issues not yet identified.

Microsoft is actively investigating ways of making C++ more usable in the kernel. It is not yet known whether all of the C++ features that can be applied to user-mode code can be made available for kernel-mode code.

•

The use of the C++ compiler as a "super-C" is typically expected to work, but such use of the compiler is at the developer's own risk.

•

It is currently impractical to identify problematic C++ constructs mechanically, so developers must carefully analyze the compiler output to ensure that the generated code is suitable for kernel mode.

•

Before committing to the use of C++, you should carefully assess whether it will work for you. In particular, you should test C++ constructs early in the development process, to ensure that the constructs do not cause the problems described in this paper or otherwise violate the principles of kernel-mode driver writing.

•

Some of the problems discussed in this paper might not become apparent until near the end of development, and that solving them might require code to be completely rewritten.

•

Several of the most insidious problems are extremely difficult to reproduce on demand while testing the driver, so a driver with an inherent unreliability might appear to run for extended periods with no problems and fail at random times. This reinforces the need for careful analysis.

It is possible to avoid many problems by careful coding and close examination of generated code. Other problems are very difficult to overcome. All of them require extra care and careful analysis on the part of the developer.

References

Resources

WHDC
Includes the Windows Driver Kit [WDK] and Windows Logo Program requirements

Whitepapers

Scheduling, Thread Context, and IRQL
Locks, Deadlocks, and Synchronization
Six Tips for Efficient Memory Use

Windows Driver Kit (WDK) Documentation

Kernel-Mode Driver Architecture Guide
In particular, see Memory Management and Driver Programming Techniques

Driver Development Tools
In particular, see Driver Verifier, GFlags, and PoolMon



Was This Information Useful?

© 2008 Microsoft Corporation. All rights reserved. Contact Us |Terms of Use |Trademarks |Privacy Statement
Microsoft
ail.com

how to create and use patch in linux

Creating a Patch File:

diff -Naur olddir newdir > new-patch
- or -
diff -Naur oldfile newfile >new-patch

A Note concerning Patch and the number of Directory Levels used:

Try to make sure when creating a patchfile that you have the same number of directories levels for both the olddir path and newdir path. For Ex:
--- old/modules/pcitable	Mon Sep 27 11:03:56 1999
+++ new/modules/pcitable Tue Dec 19 20:05:41 2000
The above diff would work ok as a patch.
--- old/try1/other/modules/pcitable	Mon Sep 27 11:03:56 1999
+++ new/modules/pcitable Tue Dec 19 20:05:41 2000
You may have problems with the above diff working as a patch. Note that the first one has 4 directory levels mentioned and the 2nd one has only two. I would suggest modifying the --- and +++ lines to make the directory structure in both to be the same number of levels deep.

Using the Patch Command:

patch -p0 <new-patch
patch -p1 <new-patch
Which one of the above patch commands you use depends on your current working directory.
Other Methods for calling the patch command - using standard input of patch:
cat new-patch | patch -p0
Other methods for calling the patch command - using a "here document":
patch -p0 << EOF
--- old/modules/pcitable Mon Sep 27 11:03:56 1999
+++ new/modules/pcitable Tue Dec 19 20:05:41 2000
@@ -1,4 +1,6 @@
0x0e11 0xae10 "cpqarray" "Compaq|Smart-2/P RAID Controller"
+0x1000 0x0010 "cpqarray" "Compaq|Integrated Array Controller"
+0x1011 0x0046 "cpqarray" "Compaq|Smart-2/P RAID Controller"
0x0e11 0xae32 "tlan" "Compaq|Netelligent 10/100"
0x0e11 0xae34 "tlan" "Compaq|Netelligent 10"
0x0e11 0xae35 "tlan" "Compaq|Integrated NetFlex-3/P"
@@ -21,6 +23,7 @@
0x1000 0x000f "ncr53c8xx" "Symbios|53c875"
0x1000 0x0012 "ncr53c8xx" "Symbios|53c895a"
0x1000 0x008f "ncr53c8xx" "Symbios|53c875J"
+0x1000 0x000a "sym53c8xx" "Symbios|53c1510"
0x1000 0x0701 "yellowfin" "Symbios|83C885 gigabit ethernet"
0x1000 0x0702 "yellowfin" "Symbios|Yellowfin G-NIC gigabit ethernet"
0x1011 0x0001 "tulip" "DEC|DECchip 21050"
--- old/usr/share/kudzu/pcitable Sun Sep 26 17:11:23 1999
+++ new/usr/share/kudzu/pcitable Tue Dec 19 20:05:41 2000
@@ -15,6 +15,8 @@
0x0e11 0x3034 "unknown" "Compaq|QVision 1280/p"
0x0e11 0x4000 "unknown" "Compaq|4000 [Triflex]"
0x0e11 0xa0f3 "ignore" "Compaq|Triflex PCI to ISA Bridge"
+0x1000 0x0010 "cpqarray" "Compaq|Integrated Array Controller"
+0x1011 0x0046 "cpqarray" "Compaq|Smart-2/P RAID Controller"
0x0e11 0xae10 "cpqarray" "Compaq|Smart-2/P RAID Controller"
0x0e11 0xae29 "unknown" "Compaq|MIS-L"
0x0e11 0xae2a "unknown" "Compaq|MPC"
@@ -46,6 +48,7 @@
0x1000 0x000f "ncr53c8xx" "Symbios|53c875"
0x1000 0x0012 "ncr53c8xx" "Symbios|53c895a"
0x1000 0x008f "ncr53c8xx" "Symbios|53c875J"
+0x1000 0x000a "sym53c8xx" "Symbios|53c1510"
0x1000 0x0701 "yellowfin" "Symbios|83C885 gigabit ethernet"
0x1000 0x0702 "yellowfin" "Symbios|Yellowfin G-NIC gigabit ethernet"
0x1000 0x0901 "unknown" "Symbios|61C102"
EOF
For the "here document", patch will read all the following lines, up until it reaches the delimeter that we specified after the << symbol. The delimeter we choose must appear at the beginning of a line, and must appear alone. In our case we are using the delimeter EOF. EOF has no special meaning, other than the fact that it is not likely that we would have a reason to type EOF at the beginning of any given line.

For more information on here documents click here:
heredoc.html

Levels in the Patch Command (-p0 or -p1?):

The -p option will optionally strip off directory levels from the patchfile. For Ex: if you have a patchfile with a header as such:
--- old/modules/pcitable	Mon Sep 27 11:03:56 1999
+++ new/modules/pcitable Tue Dec 19 20:05:41 2000
Using a -p0 will expect, from your current working directory, to find a subdirectory called "new", then "modules" below that, then the "pcitable" file below that.

Using a -p1 will strip off the 1st level from the path and will expect to find (from your current working directory) a directory called "modules", then a file called "pcitable". Patch will ignore the "new" directory mentioned in the header of the patchfile.

Using a -p2 will strip of the first two levels from the path. Patch will expect to find "pcitable" in the current working directory. Patch will ignore the "new" and "modules" directories mentioned in the header of the patchfile.

Using a -p3 in this example would not be a good thing. Patch probably wouldn't patch anything.

...

Applying a Patch:

Just change to the correct directory and give the patch command (see Using the Patch Command above). Usually is:
cd /usr/src/linux
patch -p0 <new-patch
unless the patchfile has "linux" as part of the directory structure on the +++ line, in that case issue the patch command with the -p1 as follows:
patch -p1 <new-patch

Patch Header:

In these examples a patch header begins with the two lines that begin with a --- and +++. This would indicate which filename to begin processing.

Multiple Patches in a Patchfile:

A patchfile can have multiple sections, each beginning with the --- / +++ headers mentioned above. So a single patchfile can actually modify several files. If you are reviewing a patchfile be sure to search for every instance of --- throughout the patchfile to see all the different files it patches.

An Example of one method to browse a patchfile:

less /tmp/file-patch <ENTER>
/--- <ENTER>
n
n
n
Note that the "/" invokes the search function in less, and each time you press "n" less will search for the next matching searchstring.

Hunks:

A hunk is a section to be patched. A hunk usually begins and ends with lines that don't need any changes, they are just mentioned so that Patch can find the appropriate place in the files to be patched. Hunk in these examples begins with a double at symbol (@@) and ends when either a new hunk starts or it finds a new patch header. Here's an example of a hunk:
@@ -1,4 +1,6 @@
0x0e11 0xae10 "cpqarray" "Compaq|Smart-2/P RAID Controller"
+0x1000 0x0010 "cpqarray" "Compaq|Integrated Array Controller"
+0x1011 0x0046 "cpqarray" "Compaq|Smart-2/P RAID Controller"
0x0e11 0xae32 "tlan" "Compaq|Netelligent 10/100"
0x0e11 0xae34 "tlan" "Compaq|Netelligent 10"
0x0e11 0xae35 "tlan" "Compaq|Integrated NetFlex-3/P"

Indentation in a Hunk:

In these examples you will notice that each hunk has been indented so that the text does not occupy the first column. In these examples Patch uses the first column in each patch to list the lines to be added or removed.

The First Column of a Hunk (+, -, and a blank):

The + would indicate that this particular line is to be added.

The - would indicate that this particular line is to be removed.

A line with neither a plus or minus would indicate that this particular line of code is just a reference point.

An Example Patchfile:

diff -u old/modules/pcitable new/modules/pcitable
--- old/modules/pcitable Mon Sep 27 11:03:56 1999
+++ new/modules/pcitable Tue Dec 19 20:05:41 2000
@@ -1,4 +1,6 @@
0x0e11 0xae10 "cpqarray" "Compaq|Smart-2/P RAID Controller"
+0x1000 0x0010 "cpqarray" "Compaq|Integrated Array Controller"
+0x1011 0x0046 "cpqarray" "Compaq|Smart-2/P RAID Controller"
0x0e11 0xae32 "tlan" "Compaq|Netelligent 10/100"
0x0e11 0xae34 "tlan" "Compaq|Netelligent 10"
0x0e11 0xae35 "tlan" "Compaq|Integrated NetFlex-3/P"
@@ -21,6 +23,7 @@
0x1000 0x000f "ncr53c8xx" "Symbios|53c875"
0x1000 0x0012 "ncr53c8xx" "Symbios|53c895a"
0x1000 0x008f "ncr53c8xx" "Symbios|53c875J"
+0x1000 0x000a "sym53c8xx" "Symbios|53c1510"
0x1000 0x0701 "yellowfin" "Symbios|83C885 gigabit ethernet"
0x1000 0x0702 "yellowfin" "Symbios|Yellowfin G-NIC gigabit ethernet"
0x1011 0x0001 "tulip" "DEC|DECchip 21050"
--- old/usr/share/kudzu/pcitable Sun Sep 26 17:11:23 1999
+++ new/usr/share/kudzu/pcitable Tue Dec 19 20:05:41 2000
@@ -15,6 +15,8 @@
0x0e11 0x3034 "unknown" "Compaq|QVision 1280/p"
0x0e11 0x4000 "unknown" "Compaq|4000 [Triflex]"
0x0e11 0xa0f3 "ignore" "Compaq|Triflex PCI to ISA Bridge"
+0x1000 0x0010 "cpqarray" "Compaq|Integrated Array Controller"
+0x1011 0x0046 "cpqarray" "Compaq|Smart-2/P RAID Controller"
0x0e11 0xae10 "cpqarray" "Compaq|Smart-2/P RAID Controller"
0x0e11 0xae29 "unknown" "Compaq|MIS-L"
0x0e11 0xae2a "unknown" "Compaq|MPC"
@@ -46,6 +48,7 @@
0x1000 0x000f "ncr53c8xx" "Symbios|53c875"
0x1000 0x0012 "ncr53c8xx" "Symbios|53c895a"
0x1000 0x008f "ncr53c8xx" "Symbios|53c875J"
+0x1000 0x000a "sym53c8xx" "Symbios|53c1510"
0x1000 0x0701 "yellowfin" "Symbios|83C885 gigabit ethernet"
0x1000 0x0702 "yellowfin" "Symbios|Yellowfin G-NIC gigabit ethernet"
0x1000 0x0901 "unknown" "Symbios|61C102"

Analyzing the above Example:

This example was created using the following diff command:
diff -u old/modules/pcitable new/modules/pcitable
However, now I would probably only use the -Naur options instead of just -u. This example modifies two files: new/modules/pcitable, and new/usr/share/kudzu/pcitable.

The first patch header has two hunks. These particular hunks are adding 2 lines and 1 line respectively.

Thursday, November 13, 2008

linux device driver resource

http://ubuntuforums.org/showthread.php?t=426411

Linux device drivers and modules (as the kernel itself) are written in C language.

Important documents:

Howto do kernel development. Converting to a kernel monkey.
http://lwn.net/Articles/160191/
+
http://www.kroah.com/log/linux/howto.html

Online book: Linux Device Drivers
http://www.xml.com/ldd/chapter/book/index.html but the 3.ed is more up to date. ( EDIT: online PDF edition...)

Device Driver Kit (Download this KIT before you start Open the .iso file in your file browser. Surprise, it also contains the 3.edition of ldd.
http://kerneltrap.org/node/6638

Coding in and for the kernel, coding style
http://kerneltrap.org/files/Jeremy/CodingStyle.txt

A tiny driver sample
http://www.linuxdevices.com/articles/AT3711365653.html

There have been some tests on using C++ in the kernel, but honestly, it doesn't fit in.
http://netlab.ru.is/exception/LinuxCXX.shtml
#########
Linux kernel factory (the hot core revealed )
http://lug.oregonstate.edu/projects/kernelmap/map.php

I collected these links from my super chaotic kernel page... BE WARNED !!

Other importante programming resources....

Tuesday, November 11, 2008

how to bind a irq to a specified CPU

ail.comcat cpu_mask > /proc/irq/n/smp_affinity
cpu_mask is a 32-bit mask
n is the interrupt vector

Monday, October 20, 2008

Downloading an Entire Web Site with wget

http://www.linuxjournal.com/content/downloading-entire-web-site-wget

If you ever need to download an entire Web site, perhaps for off-line viewing, wget can do the
job—for example:

$ wget \
--recursive \
--no-clobber \
--page-requisites \
--html-extension \
--convert-links \
--restrict-file-names=windows \
--domains website.org \
--no-parent \
www.website.org/tutorials/html/

This command downloads the Web site www.website.org/tutorials/html/.

The options are:

  • --recursive: download the entire Web site.

  • --domains website.org: don't follow links outside website.org.

  • --no-parent: don't follow links outside the directory tutorials/html/.

  • --page-requisites: get all the elements that compose the page (images, CSS and so on).

  • --html-extension: save files with the .html extension.

  • --convert-links: convert links so that they work locally, off-line.

  • --restrict-file-names=windows: modify filenames so that they will work in Windows as well.

  • --no-clobber: don't overwrite any existing files (used in case the download is interrupted and
    resumed).

Wednesday, September 24, 2008

compress decompress files in Linux

http://linuxtuts.blogspot.com/2008/02/handling-archives-in-linux.html
 How to compress/decompress files in Linux.
Zip files
To compress of decompress zip files you must have the following programs installed in your computer

zip to compress files
unzip to decompress files

here are the commands
to zip a single file

$zip archive_name.zip file_name 

to zip a single folder use the folder name instead of file_name

$zip -r archive_name.zip folder_name 

to zip multiple files use

$zip archive_name.zip file1 file2 file3 file4 file5 

you can also zip a combination of files and folders into one zip file like this

$zip -r archive_name.zip file1 file3 file4 folder_name1/ folder2/ folder3/ 

to unzip a file u need unzip program installed in your system

$unzip file.zip

to view data in the zip file use

$unzip -v file.zip

Rar files
You need the rar program from rarlabs.com to compress/decompress these files or you can use unrar or unrar-nonfree packages

$rar -a file.rar /folder/

to uncompress use

$rar -x file.rar

to view the data in the file use

$rar -v file.rar

Tar Files
to compress files into a tar archive use

$tar cvf file.tar folder/ *

to unpack or to uncompress us

$tar -xvf file.tar

to view data in the tar file use

$tar -tvf file

Here the 'v' is optional, it just gives verbose output

Tar.gz files
to compress into .tar.gz use

$tar czvf file.tar.gz folder

to uncompress use

$tar xzvf file.tar.gz

to view data in the tar.gz file use

tar tzvf file.tar.gz

bz2 files
to compress into bz2 format use bzip2

$bzip2 file

above command will create an archive named file.bz2. Also bz2 works on only files.
So if you want to compress a folder you have to pack it into a tar file first and use bzip2 on it.
to uncompress or unpack use

$bzip2 -d file.bz2

tar.bz2 files
to compress into a tar.bz2 file use

$tar cjvf file.tar.bz2 folder

to uncompress tar.bz2 files use

$tar -xjvf file.tar.bz2

to view data in the file use

$tar -tjvf file.tar.bz2

Friday, September 19, 2008

gentoo install from harddisk

dig from gentoo install guide
http://www.gentoo.org/doc/en/gentoo-x86-quickinstall.xml

grub load untared minimal live cd

======

1. for windows system (fat32): install grub for dos
cp grld to C:\
add "c:\grldr="grub"" to the last of boot.ini file

2. configure menu.lst
need to uncompress *.sqashfs folder to partition
uncompress isolinux folder
set it to :

kernel (hd0,0)/boot/gentoo/gentoo root=/dev/ram0 init=/linuxrc dokeymap
looptype=squashfs loop=/image.squashfs cdroot vga=791
splash=silent,theme:livecd-2007.0 CONSOLE=/dev/tty1 nox
initrd (hd0,0)/boot/gentoo/gentoo.igz


c:\boot.ini
c:\grldr
c:\menu.lst
c:\boot\gentoo\gentoo
c:\boot\gentoo\gentoo.igz
x:\livecd
x:\image.squashfs


if it is grub for linux , just need to set menu.lst file correctly.


(The following specs and the timing information should help you determine
a rough estimate of the time you need to complete your install)


# grep bogo /proc/cpuinfo
bogomips : 3337.81

# hdparm -tT /dev/sda
/dev/sda:
Timing cached reads: 1100 MB in 2.00 seconds = 549.97 MB/sec
Timing buffered disk reads: 224 MB in 3.01 seconds = 74.36 MB/sec

# grep MemTotal /proc/meminfo
MemTotal: 509248 kB


livecd root # ifconfig eth0 192.168.1.10/24
livecd root # route add default gw 192.168.1.1
livecd root # echo nameserver 192.168.1.1 > /etc/resolv.conf



(ext2 is all you need on the /boot partition)
livecd ~ # mke2fs /dev/sda1

(Let's use ext3 on the main partition)
livecd ~ # mke2fs -j /dev/sda3

(Create and activate swap)
livecd ~ # mkswap /dev/sda2 && swapon /dev/sda2



livecd ~ # mount /dev/sda3 /mnt/gentoo
livecd ~ # mkdir /mnt/gentoo/boot
livecd ~ # mount /dev/sda1 /mnt/gentoo/boot
livecd ~ # cd /mnt/gentoo


Howto make partition changes visible to the kernel without reboot

(very important if you cannot reboot your system but changed partition during installation )

Install parted in debian

#apt-get install parted

Now this will install partprobe and parted.Partprobe inform the OS of partition table changes. partprobe is a program that informs the operating system kernel of partition table changes, by requesting that the operating system re-read the
partition table.

Syntax

partprobe [-d] [-s] [devices...]

-d Donât update the kernel.
-s Show a summary of devices and their partitions.
-h Show summary of options.
-v Show version of program.

Using partprobe

#partprobe

(Check the clock)
livecd gentoo # date
Mon Mar 6 00:14:13 UTC 2006

(Set the current date and time if required)
livecd gentoo # date 030600162006 (Format is MMDDhhmmYYYY)
Mon Mar 6 00:16:00 UTC 2006

livecd gentoo # links http://www.gentoo.org/main/en/mirrors.xml
(Pick a mirror, move to the releases/x86/2008.0/stages/ directory, highlight the
stage3 of your choice, probably the i686 stage3 and press D to download it)


(Or download it directly with wget without choosing a nearby mirror)
livecd gentoo # wget ftp://gentoo.osuosl.org/pub/gentoo/releases/x86/2008.0/stages/stage3-i686*tar.bz2

livecd gentoo # time tar xjpf stage3*

real 1m14.157s
user 1m2.920s
sys 0m7.530s

livecd gentoo # cd /mnt/gentoo/usr
livecd usr # links http://www.gentoo.org/main/en/mirrors.xml
(Pick a mirror, move to the snapshots/ directory,
highlight portage-latest.tar.bz2 and press D to download it)


(Or download it directly with wget without choosing a nearby mirror)
livecd gentoo # cd /mnt/gentoo/usr
livecd usr # wget http://gentoo.osuosl.org/snapshots/portage-latest.tar.bz2

livecd usr # time tar xjf portage-lat*

real 0m40.523s
user 0m28.280s
sys 0m8.240s

livecd usr # cd /
livecd / # mount -t proc proc /mnt/gentoo/proc
livecd / # cp -L /etc/resolv.conf /mnt/gentoo/etc/
livecd / # chroot /mnt/gentoo /bin/bash
livecd / # env-update && source /etc/profile
>>> Regenerating /etc/ld.so.cache...


livecd / # ls /usr/share/zoneinfo
(Using Brussels as an example)
livecd / # cp /usr/share/zoneinfo/Europe/Brussels /etc/localtime

livecd / # date
Wed Mar 8 00:46:05 CET 2006

livecd / # cd /etc
livecd etc # echo "127.0.0.1 mybox.at.myplace mybox localhost" > hosts
livecd etc # sed -i -e 's/HOSTNAME.*/HOSTNAME="mybox"/' conf.d/hostname
(Use defined host name and check)
livecd etc # hostname mybox
livecd etc # hostname -f
mybox.at.myplace



livecd etc # time emerge gentoo-sources

real 2m51.435s
user 0m58.220s
sys 0m29.890s
livecd etc # cd /usr/src/linux
livecd linux # make menuconfig
(Configure your kernel)
livecd linux # time make -j2

(Elapsed time depends highly on the options you selected)
real 3m51.962s (so fast ........ impossible .... -_-! )
user 3m27.060s
sys 0m24.310s





livecd linux # make modules_install
livecd linux # cp arch/i386/boot/bzImage /boot/kernel


livecd linux # cd /etc
livecd etc # nano -w fstab
/dev/sda1 /boot ext2 noauto,noatime 1 2
/dev/sda3 / ext3 noatime 0 1
/dev/sda2 none swap sw 0 0



livecd etc # cd conf.d
livecd conf.d # echo 'config_eth0=( "192.168.1.10/24" )' >> net
livecd conf.d # echo 'routes_eth0=( "default via 192.168.1.1" )' >> net
livecd conf.d # rc-update add net.eth0 default
(If you compiled your network card driver as a module,
add it to /etc/modules.autoload.d/kernel-2.6)

livecd conf.d # echo r8169 >> /etc/modules.autoload.d/kernel-2.6
(If you want to reconnect via ssh after you have rebooted your new box)
livecd conf.d # rc-update add sshd default




livecd conf.d # passwd
New UNIX password: type_the_password
Retype new UNIX password: type_the_password_again
passwd: password updated successfully



livecd conf.d # nano -w /etc/conf.d/clock
TIMEZONE="Europe/Brussels"


livecd conf.d # nano -w /etc/rc.conf
livecd conf.d # nano -w /etc/conf.d/rc
livecd conf.d # nano -w /etc/conf.d/keymaps





livecd conf.d # time emerge syslog-ng vixie-cron

real 1m54.099s
user 1m2.630s
sys 0m34.620s
livecd conf.d # rc-update add syslog-ng default
livecd conf.d # rc-update add vixie-cron default


livecd conf.d # emerge xfsprogs (If you use the XFS file system)
livecd conf.d # emerge jfsutils (If you use the JFS file system)
livecd conf.d # emerge reiserfsprogs (If you use the Reiser file system)
livecd conf.d # emerge dhcpcd (If you need a DHCP client)
livecd conf.d # emerge ppp (If you need PPPoE ADSL connectivity)

livecd conf.d # time emerge grub

real 1m4.634s
user 0m39.460s
sys 0m15.280s
livecd conf.d # nano -w /boot/grub/grub.conf


default 0
timeout 10

title Gentoo
root (hd0,0)
kernel /boot/kernel root=/dev/sda3



livecd conf.d # grub
Probing devices to guess BIOS drives. This may take a long time.

grub> root (hd0,0)
Filesystem type is ext2fs, partition type 0xfd

grub> setup (hd0)
Checking if "/boot/grub/stage1" exists... yes
Checking if "/boot/grub/stage2" exists... yes
Checking if "/boot/grub/e2fs__5" exists... yes
Running "embed /boot/grub/e2fs_stage1_5 (hd0)"... 16 sectors are embedded.
succeeded
Running "install /boot/grub/stage1 (hd0) (hd0)1+16 p (hd0,0)/boot/grub/stage2 /boot/
grub/menu.lst"... succeeded
Done.

grub> quit


livecd conf.d # exit
livecd / # umount /mnt/gentoo/proc /mnt/gentoo/boot /mnt/gentoo
livecd / # reboot
(Don't forget to remove the CD)



(Clean up your known_hosts file because your new box
has generated a new definitive hostkey)

$ nano -w ~/.ssh/known_hosts
(Look for the IP of your new PC and delete the line,
then save the file and exit nano)


(Use the IP address of your new box)
$ ssh root@192.168.1.10
The authenticity of host '192.168.1.10 (192.168.1.10)' can't be established.
RSA key fingerprint is 96:e7:2d:12:ac:9c:b0:94:90:9f:40:89:b0:45:26:8f.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.1.10' (RSA) to the list of known hosts.
Password: type_the_password

mybox ~ # adduser -g users -G lp,wheel,audio,cdrom,portage,cron -m john
mybox ~ # passwd john
New UNIX password: Set John's password
Retype new UNIX password: Type John's password again
passwd: password updated successfully

mybox ~ # emerge mirrorselect
mybox ~ # mirrorselect -i -o >> /etc/make.conf
mybox ~ # mirrorselect -i -r -o >> /etc/make.conf
(Usually, (the number of processors + 1) is a good value)
mybox ~ # echo 'MAKEOPTS="-j2"' >> /etc/make.conf


mybox ~ # emerge -vpe world
(Portage displays the packages and their USE flags, as an example, let's
disable ipv6 and fortran, and enable unicode)

mybox ~ # echo 'USE="nptl nptlonly -ipv6 -fortran unicode"' >> /etc/make.conf

mybox ~ # cd /etc
mybox etc # nano -w locale.gen
mybox etc # locale-gen

mybox etc # nano -w make.conf
(Set -march to your CPU type in CFLAGS)
CFLAGS="-O2 -march=athlon-xp -pipe"
(Add the following line)
FEATURES="ccache"
(Only add the following if you know what you're doing)
ACCEPT_KEYWORDS="~x86"

(Install ccache)
mybox etc # emerge ccache

(Please note that the switch to ~x86 causes many packages to be upgraded)
mybox etc # emerge -vpuD --newuse world
(Take a good look at the package list and their USE flags,
remove blocking packages if any, and start the lengthy process)

mybox etc # time emerge -vuD --newuse world
(79 packages have been (re)compiled)

real 180m13.276s
user 121m22.905s
sys 36m31.472s


(Remerge libtool to avoid further potential problems)



mybox etc # emerge --oneshot libtool

(Update config files, make sure you do not let dispatch-conf
update config files you have edited)

mybox etc # dispatch-conf

(If perl has been updated, you should run the perl-cleaner script)
mybox etc # time perl-cleaner all
real 1m6.495s
user 0m42.699s
sys 0m10.641s



(In case of a major upgrade of python, you should run the python-updater script)
mybox etc # python-updater