Search and replace with Windows PowerShell

I was trying to re-brand the source code of our product. It involved replacing the internal code name of the project with the actual product name. 

If you ever had to do some sort  search and replace across a bunch of files with a certain level of control, here is a simple PowerShell script you can use. For example, lets say the task is to replace all instances of "CodeName" with "ProductName" across all files in the current directory. 

Get-Childitem . -rec -name -include *.proj | ForEach-Object { (Get-Content $_) | ForEach-Object {$_ -replace "CodeName", "ProductName" } |Set-Content $_ }

It may seem confusing at once, but there are very consistent simple and easy to follow rules that PoweShell follows. All commands (called cmdlets) in PowerShell are in the form of verb + noun. Just like the first command above Get-ChildItem, there is Get-Process, Get-Content etc The first command 

Get-Childitem . -rec -name -include *.proj 

recursively gets a list of all file names that match *.proj in the directories below the current one.

The output of this command is piped to the next command which is a bit complex

ForEach-Object { (Get-Content $_) | ForEach-Object {$_ -replace "CodeName", "ProductName" } | Set-Content $_ }

The ForEach-Object {...} command simply does 'something' for each object it receives as input from the pipe. The first command sent the list of file names, so its going to do 'something' to this list of names.

This 'something' is another set of commands.

(Get-Content $_) | ForEach-Object {$_ -replace "CodeName", "ProductName" | Set-Content $_ 

Here the $_ represents the current object; a file name with complete path in this case. Get-Content $_ gets the content of the file and pipes the output to another ForEach-Object, which replaces all the instances of string "CodeName" with "ProductName" for each line of the current file's (represented by $_) content.

The replaced file is then written to disk using Set-Content $_ 

Heres another one I wrote to help me automate 'svn move' for files

gci -rec | where { !$_.PsIsContainer } | where { $_.Name.Contains("CodeName") } | foreach { svn move $_.FullName ($_.DirectoryName + "\" + ($_.Name -creplace "CodeName", "ProductName")) }

Here 'gci' and 'foreach' are simply aliases for the more verbose 'Get-ChildItem' and 'ForEach-Object' commands.

Apart from such basic use, PowerShell offers a very impressive set of functions to work with on your machine and well as remote ones, all without having to click-click-click :) Though its mainly oriented towards admin type tasks, I am sure devs can equally exploit its functionality in day to day use.

You can download PowerShell from here

http://blogs.msdn.com/powershell/pages/download-windows-powershell.aspx

Tip: Quickly delete all unversioned SVN files

I was looking for a quick way to delete all my unversioned SVN files. If you have PowerShell, I came across this very simple command

(svn status --no-ignore) -match '^\?' -replace '^\?\s+' | rm

It finds all matches from the out put of ‘svn status’ command, and then replaces the ‘?’ followed by bunch of white spaces (\s+) to get you the list of files that are unversioned. These files are then removed with the ‘rm’ command.

Comparing Versions in .NET

A quick note when comparing System.Version objects. Looking at this expression, my guess would be that it is true, but it is not..

new Version("9.1") == new Version("9.1.0.0") // FALSE

What’s more weird is that this expression below is true!

new Version("9.1") < new Version("9.1.0.0") // TRUE!

And that’s because unspecified components of the version are set to –1. In this case the build and revision for the Version on the left-hand side is –1.

So if you plan to use short-form versions like “9.1” you have to be careful when comparing it with full versions of the form “9.1.3”. This is specially important when one of the versions is constructed dynamically (user input or by querying some database).

Sybase user ‘alias’

In the last post I stated that a Sybase ‘login’ is mapped to a ‘database user’. In this post, I’ll elaborate on it further and introduce the concept of an ‘alias’.

One ‘login’ can be mapped to only one ‘database user’ in the given database. Obviously, at the server-level one ‘login’ can, and generally is mapped to many database users in different databases. For example, the ‘sa’ login is mapped to ‘dbo’ users in all databases. But how about the relationship in the other direction i.e. from ‘database user’ to a ‘login’?

Each ‘database user’ is also associated with one ‘login’ – except when there exists an ‘alias’. It is possible for two or more logins to be mapped to the same ‘database user’ by creating an ‘alias’. This allows both the logins to act as the same ‘database user’. This is typically done to allow multiple logins to act as database owners (i.e. user ‘dbo’). For example, as I mentioned earlier, ‘sa’ is always mapped to the ‘dbo’ user. If you wish to make someone else, lets say, ‘president’ login act as the ‘dbo’ database user, you create an alias

sp_addalias ‘president’, ‘dbo’

This would make both ‘sa’ and ‘president’ act as ‘dbo’. Note that you can have many aliases for a given ‘login’. You could very well also alias ‘vice_president’ to ‘sa’

sp_addalias ‘vice_president’, ‘dbo’

Even though multiple logins now act as the same ‘database user’, their activity can still be audited individually. This would not be possible if you simply gave the password for ‘sa’ login to everyone who you wish to make ‘dbo’, for example.

Sybase ‘guest’

A little background on Sybase first. Each instance of Sybase server typically contains many databases. TCP/IP is usually enabled and 5000 is the default port, but this can obviously be changed. One logons to the Sybase instance and thereby gets potential access to all databases residing at the instance. Some of the databases like ‘master’, ‘tempdb’, ‘model’ etc get created at install time by the Sybase installer and are required for Sybase to run properly.

Logins - A ’login’ gives you access to the instance and is a server-level concept. You can add a ‘login’ using

sp_addlogin ‘login_name’, ‘password’

This adds an entry to the ‘master.dbo.syslogins’ table. Although a ‘login’ gives you access to the instance, to get access to a particular database, it must be mapped to an existing ‘database user’.

Database Users - Each database maintains its own set of database users. When creating a ‘database user’, the associated ‘login’ has to be supplied – like this

sp_adduser ‘login_name’, ‘database_user’

This adds an entry to the ‘<database_name>.dbo.sysusers’ table. Here the ‘login_name’ is mapped to ‘database_user’. Now, one thing to note is that one may add a ‘login’ but never map it to any particular ‘database user’ – what happens then is a little tricky and depends on the existence of a special user called ‘guest’.

Special ‘guest’ - If an explicit mapping does not exist between the ‘login’ and any of the existing database users, then the ‘login’ is implicitly mapped to the special ‘guest’ user, if one exists. If the 'guest’ user does not exist, the ‘login’ is denied access to the database. One can easily add the ‘guest’ user if it does not exist by using

sp_adduser guest

When creating new database (using ‘create database’ for example), the ‘model’ database gets used as a template. In the versions of Sybase (12.5, 15, 15.5 on Linux/Windows) I played with I did not see the user ‘guest’ defined in the ‘model’ database. So the new databases based on ‘model’ will also not have the ‘guest’ user by default.

MacPorts

I was looking for an easy way to upgrade an outdated version 2.6.9 of rsync on my OS X Lion MacBook. I'm not too good with UNIX to begin with and finding required dependencies etc on my OS X Lion would be a time consuming task. So I was looking around for any ready-to-use installers or binaries of the latest 3.0.8, when I stumbled upon this neat project called MacPorts. It provides an easy to use system for getting binaries for such open-source tools on the Mac OS X systems. Installation was simple - this is from the MacPorts website

- “dmg” disk images for Lion and Snow Leopard. These contain pkg installers for use with the Mac OS X Installer. This is the simplest installation procedure that most users should follow after meeting the requirements listed below. An installer forLeopard, as a legacy platform, is also available.

Before installing enable the root user and install Xcode from the AppStore (on Lion) or Apple dev connection website (Snow Leopard/Leopard). You would need X11 as well for ports that depend on it. Use the package installer and then simply run this to update to the latest of MacPorts

sudo port -v selfupdate

Now, if you know the program you want to install is supported by MacPorts, like rsync for example, all you do to install it is

sudo port install rsync

 

Enable root user on OS X Lion

1. Launch the Directory Utility. It is located in /System/Library/CoreServices.
2. Click the padlock icon on the bottom left corner and enter your Administrator login credentials to unlock the application
3. In the Edit menu, select Enable Root User
Again from the Edit menu, select Change Root Password. A dialog window will apear prompting you to enter a root password

Remote unauthenticated DoS attack in Oracle

About 3 years ago, I had reported a possible remote DoS attack in Oracle. Oracle addressed the issue and acknowledged me for the discovery in its Critical Patch Update CPU April 2009 . Specifically it is the CVE-2009-0991 entry in the Oracle Database Risk Matrix. We never created any formal public advisory for this, but I was looking for something to post on my blog and writing about it seemed like a good idea :)

There is some flaw in the way Oracle handles user authentication. The authentication process seems to involve an initial TNS CONNECT from the client to which the server responds with an ACCEPT assuming correct server IP and port (1521 is the default LISTENER port) and SID/service name is specified. After which is negotiation of security services and exchange of a couple more packets. At one point, the client seems to send a login packet with username to which the server responds with a key (AUTH_SSKEY) indicating that some sort of a session initiation has begun.  This is done whether the username specified in the packet is valid or not. If the client then abruptly terminates the connection as soon as the session key packet is received, the database seems to still keep track of that session internally. On Oracle 10g, sending such multiple login packets with invalid/arbitrary usernames and abruptly terminating connections after the AUTH_SSKEY is received, will very soon exhaust the 'maximum concurrent allowed sessions' limit and thus cause a DoS for other users. On Oracle 9i R2, I could get same results, but the login packet needs a valid username, because in 9i R2 the session key is only generated for valid users. Nevertheless, one can always use known usernames like 'sys', 'dbsnmp' etc

Oracle later confirmed this to be an issue in 11gR1 as well. To resolve the issue, Oracle enhanced the INBOUND_CONNECT_TIMEOUT configuration parameter in CPU April 2009.  This parameter allows one to specify a timeout value in seconds to wait until the initiated login process is completed. So, using a small enough value for this parameter, an attacker will not be able to create too many such half-open logon sessions in order to cause the DoS. The default  timeout is 60 sec, but it should be less IMHO, although it would really depend on many factors specific to each setup.

New error codes ORA-3135 and ORA-3136 were also added to indicate when the timeout is exceeded, in addition to the existing codes.

Rights to this research belong to my company Application Security, Inc.

Revoke in Sybase

I learned something new today about how Sybase's  revoke SQL statement works. In Sybase a database user can be granted a privilege directly or indirectly (via role and/or group membership). In case of a direct grant, revoking the privilege will take the privilege out. But in case of an indirect grant its a little different.

If the indirect grant comes via a role, then revoke  doesn't do anything (actually throws an error), but if the grant comes from the group to which the user belongs, then the user would be denied  the privilege. Everyone else belonging to the group would continue to have the privilege except this user. A row is added to the sysprotects table to this effect.  Also note it should be noted that this applies to object privileges only - revoking an indirectly granted system privilege (either via role or group) does not have any effect.

Also, its worth nothing here that role privileges trump any privilege grants and revokes at the group or direct user level.

To explain this by an example, lets assume user user1 belongs to group group1 and is also granted a role role1.

grant select on table table1 to group1

This gives user1  a select privilege on table1 via group group1.

revoke select on table1 from user1

This denies user1 select on table1

grant select on table1 to role1

This gives the select privilege back to user1 because the role privilege trumps any group and user level grants/revokes

 

System-reserved roles in DB2 version 9.7

DB2 9.7 added a few built-in roles like SYSROLE_AUTH_DBADM, SYSROLE_AUTH_SECADM.  These roles show up in the syscat.roles and syscat.roleauth catalog views as regular roles, but DB2 treats them as   special roles reserved exclusively for certain database authorities. For example, SYSROLE_AUTH_DBADM is granted automatically whenever DBADM is granted to someone. And it is revoked automatically when DBADM is revoked. These  roles can not be granted/revoked manually (using grant/revoke statement for example). These roles hold system chosen object privileges. For example, SYSROLE_AUTH_DBADM role hold EXECUTE privilege over many (all?) procedures/functions under the SYSPROC, SYSIBMADM, SYSIBMINTERNAL schemas.

More DBADM magic in DB2 9.7

Previously (below version 9.7) granting DBADM granted all the database authorities explicitly - specifically these ones

BINDADD
CONNECT
CREATETAB
CREATE_EXTERNAL_ROUTINE
CREATE_NOT_FENCED_ROUTINE
IMPLICIT_SCHEMA
QUIESCE_CONNECT
LOAD

And when you revoked DBADM, these guys would still linger on! In 9.7 this changes - granting DBADM still gives you all these authorities ..but implicitly. You will not see them in the system catalog SYSCAT.DBAUTH view being granted. Nevertheless you will have them. And, naturally, when you revoke DBADM, they will be gone along with DBADM - just the way it should have always been..

DBADM and ACCESSCTRL

Continuing along the same lines as the previous post about DBADM and DATAACCESS, there is much more that was taken away from DBADM in DB2 9.7.

1.  ability to GRANT/REVOKE *any* database authority (except SECADM, which it never had)

2. ability to GRANT/REVOKE *any* privilege on *any* database object

This again is a significant step towards achieving separation of duties between the 'security administrator' and the 'database administrator'. A new authority 'Access Control authority' (ACCESSCTRL) was introduced to hold these privileges. By default, when DBADM is granted, ACCESSCTRL is also granted, but one can now choose to not do so by using

GRANT DBADM WITHOUT ACCESSCTRL

One thing to keep in mind is when DBADM is revoked, DATAACCESS and ACCESSCTRL (if granted by default) are NOT automatically revoked.

Whats with DBADM in DB2 9.7

DB2 seems to have taken the 'separation of duties' principal to heart. This is really good from a security point of view. Starting with 9.7, the database administrator authority (DBADM), by itself, no longer has implicit access to DB2 data! It means, admins can no longer *see* sensitive data (this credit card nos..) stored in the database.
Another authority called DATAACCESS was introduced which does have implicit access to data. When granting someone DBADM, the default is still to grant DATAACCESS, but this can be changed using

GRANT DBADM WITHOUT DATAACCESS

 

DB2 9.7 SECADM

I mentioned in the previous post that the 'security administrator' authority (SYSADM) no longer has ability to grant/revoke DBADM and SECADM. Only SECADM now has the ability to grant DBADM and SECADM. In fact, in 9.7 *only* SECADM can grant/revoke *all* database authorities.

A new 'access control' authority (ACCESSCTRL) is introduced in 9.7 which grants one the ability to grant/revoke *all* database authorities and *all* privileges on *all* objects! This new authority is implicitly included in the SECADM authority.

C++/CLI : Accessing managed types from unmanaged code

I just found this long blog post I wrote at my work place last year - thought its worthwhile sharing it Date: 1/21/2010 Introduction This article aspires to explain one of the ways a managed type (/clr compiled) can be used/invoked from a native type (/clr compiled) and more importantly from unmanaged C/C++ code. Background  When I started dabbling in C++/CLI, I often got confused by these words like managed code, unmanaged code, managed type, native types etc, so for someone who happens to be in the same boat, lets try to get that out of the way. Simply put, managed code would be any code that is compiled with the /clr (and its variants like /clr:pure) flag, and Unmanaged code would be the one that is not compiled with /clr (or any of its variants). The concept of managed and native types applies to managed code only. There are no such concepts in the unmanaged world. Everything is "native" in unmanaged code... but the types are not referred to "native types" as such. Again, simply put, a Managed type is a type that is defined with the "ref" keyword (ManagedType in Listing 1 below, for example). A Native type is the one that is declared without the "ref" keyword (NativeType in Listing 2, for example). Managed types are allocated on managed stack or managed heap. Native types are allocated on native stack or native heap. The managed and native stack are much similar as far as the way memory gets cleaned up. Once the object goes out of scope, the memory is reclaimed. Memory in managed heap is managed by the garbage collector. Memory in native heap is managed by the programmer. Accessing Managed type from Native type  First, lets briefly see how to access a managed type from a native type, but the main purpose of this post is to see how one can access a managed type from purely unmanaged code (compiled without /clr). The method is sort of explained here (http://msdn.microsoft.com/en-us/magazine/cc300632.aspx) but I found it somewhat difficult to follow so here is my attempt to simplify it a little.  As an example, lets invoke the DoSomething method of ManagedType from NativeType, and then invoke the same method from a global UnmanagedFunction (see Listing 6)  Listing 1 - ManagedType.h public ref class ManagedType { void DoSomething(); };   To use the ManagedType from NativeType, one could simply try to include the reference type as a member of native type as shown in Listing 1.1 below. It will cause a compiler error. Native types are allocated on native stack/heap but managed types are not. Including a managed type as a member would mean that it will also get allocated on the native stack/heap, which would violate the C++/CLI design. Listing 1.1 - NativeType.h class NativeType { public: ManagedType^ managedType; // error...not allowed }  Operator overloads make its use much the same as the underlying managed type it wraps. See the implementation of NativeType in Listing 3, for example. Listing 2 - NativeType.h #include <vcclr.h> class NativeType { public: NativeType(); ~NativeType(); void proxy(); gcroot<ManagedType^> virtualManagedType; }   Listing 3 - NativeType.cpp (compile with /clr) #include "NativeType.h" NativeType::NativeType() { //one way of assigning managed reference to gcroot<T> handle virtualManagedType = gcnew ManagedType(); } void NativeType::proxy() { if (virtualManagedType) { virtualManagedType->DoSomething(); } }  As you can see virtualManagedType can be treated just the same as a reference to ManagedType.  Accessing Managed type from Unmanaged code   Now, lets see how to access the ManagedType from unmanaged code. It has no idea of what ManagedType is, but how about a NativeType? After all, isn't native type a simple C/C++ type albeit compiled with /clr ? The answer is not really...  Listing 4 - UnmanagedCode.cpp (compile without /clr) #include "NativeType.h" // error... ...  Doing so produces an error. The reason is the gcroot<T> member of NativeType. Remember that it wraps GCHandle which is a managed type, and unmanaged code has no idea of what that means.  Here, we can use the _MANAGED precompiler directive and intptr_t to trick the unmanaged code  into believing that the referenced type is an integer type (unmanaged code) called intptr_t. _MANAGED is set when the compilation mode is /clr. We wrap the gcroot<T> handle in _MANAGED to make it invisible to unmanaged code, and expose an intptr_t instead. Listing 5 - NativeType.h #include <vcclr.h> class NativeType { public: NativeType(); ~NativeType(); void proxy(); #ifdef _MANAGED gcroot<ManagedType^> virtualManagedType; #else intptr_t virtualManagedType; #endif }  We can replace the gcroot<T> handle with intptr_t because the memory semantics for both are exactly the same. The intptr_t (http://msdn.microsoft.com/en-us/library/323b6b3k.aspx) is simply an integer whose size is either 32-bit or 64-bit depending on the platform. The gcroot<T> has a void pointer (void* _handle) as its only data member, which is also 32-bit or 64-bit integer depending on the platform.  Now, we can simply include NativeType.h anywhere and use NativeType  Listing 6 - UnmanagedCode.cpp (compile without /clr) #include "NativeType.h" void UnmanagedFunction() { NativeType nativeType; nativeType.proxy(); // calls ManagedType::DoSomething() } Its important to keep it in mind that what we did with the _MANAGED and intprt_t is just a trick to get the unmanaged code to "know" about the a member of NativeType called virtualManagedType that occupies a certain size memory, so that it can compile and link correctly. In both cases, the linker links to the same implementation of NativeType which is in fact managed code (remember it is /clr compiled)

DB2 9.7 SYSADM

I've been researching DB2's authorization model at work lately, so wanted to share some things I've been finding along the way.

Something I just noticed is that in 9.7,  the SYSADM (system administrator) authority no longer gets an implicit DBADM (database administrator) authority as it did in 9.1 and 9.5! It means that the SYSADM will no longer have access to any data in the database (unless it is the creator of the database as well, in which case DBADM is automatically granted)

Whats more.. the ability to grant/revoke DBADM and SECADM (security admin) no longer stays with SYSADM. And it only makes sense from point of view of 'separation of duties'