Home Page
Archive > Posts > Tags > Cygwin
Archive > Posts > Tags > Cygwin
Search:
RABiD BUNNY FEVER
K.T.K

Warning: you do not have javascript enabled. This WILL cause layout glitches.

Ping Connectivity Monitor

The following is a simple bash script to ping a different domain once a second and log the output. By default, it pings #.castledragmire.com, where # is an incrementing number starting from 0.

The script is written for Cygwin (See the PING_COMMAND variable at the top) but is very easily adaptable to Linux.

The log output is: EPOCH_TIMESTAMP DOMAIN PING_OUTPUT


#This uses Window's native ping since the Cygwin ping is sorely lacking in options
#"-n 1"=Only runs once, "-w 3000"=Timeout after 3 seconds
#The grep strings are also directly tailored for Window's native ping
PING_COMMAND=$(
	echo 'C:/Windows/System32/PING.EXE -n 1 -w 3000 $DOMAIN |';
	echo 'grep -iP "^(Request timed out|Reply from|Ping request could not find)"';
)

i=0 #The subdomain counter
STARTTIME=`date +%s.%N` #This holds the timestamp of the end of the previous loop

#Infinite loop
while true
do
	#Get the domain to run. This requires a domain that has a wildcard as a primary subdomain
	DOMAIN="$i.castledragmire.com"

	#Output the time, domain name, and ping output
	echo `date +%s` "$DOMAIN" $(eval $PING_COMMAND)

	#If less than a second has passed, sleep up to 1 second
	ENDTIME=`date +%s.%N`
	SLEEPTIME=$(echo "1 - ($ENDTIME - $STARTTIME)" | bc)
	STARTTIME=$ENDTIME
	if [ $(echo "$SLEEPTIME>0" | bc) -eq 1 ]; then
		sleep $SLEEPTIME
		STARTTIME=$(echo "$STARTTIME + $SLEEPTIME" | bc)
	fi

	#Increment the subdomain counter
	let i+=1
done
Weird filename encoding issues on windows

So somehow all of the file names in my Rammstein music directory, and some in my Daft Punk, had characters with diacritics replaced with an invalid character. I pasted one of such filenames into a hex editor to evaluate what the problem was. First, I should note that Windows encodes its filenames (and pretty much everything) in UTF16. Everything else in the world (mostly) has settled on UTF8, which is a much better encoding for many reasons. So during some file copy/conversion at some point in the directories’ lifetime, the file names had done a freakish (utf16*)(utf16->utf8) rename, or something to that extent. I had noticed that all I needed to do was to replace the first 2 bytes of the diacritic character with a different byte. Namely “EF 8x” to “Cx”, and the rest of the bytes for the character were fine. So if anyone ever needs it, here is the bash script.

LANG=;
IFS=$'\n'
for i in `find -type f | grep -P '\xEF[\x80-\x8F]'`; do
	FROM="$i";
	TO=$(echo "$i" | perl -pi -e 's/\xEF([\x80-\x8F])/pack("C", ord($1)+(0xC0-0x80))/e');
	echo Renaming "'$FROM'" to "'$TO'"
	mv "$FROM" "$TO"
done

I may need to expand the range beyond the x80-x8F range, but am unsure at this point. I only confirmed the range x82-x83.

Missing phar wrapper

Phar files are PHP’s way of distributing an entire PHP solution in a single package file. I recently had a problem on my Cygwin PHP server that said “Unable to find the wrapper "phar" - did you forget to enable it when you configured PHP?”. I couldn’t find any solution for this online, so I played with it a bit.

The quick and dirty solution I came up with is to include the phar file like any normal PHP file, which sets your current working directory inside of the phar file. After that, you can include files inside the phar and then change your directory back to where you started. Here is the code I used:

if(preg_match('/^(?:win|cygwin)/i', PHP_OS))
{
    $CWD=getcwd();
    require_once('Scripts/PHPExcel.phar');
    require_once('PHPExcel/IOFactory.php');
    chdir($CWD);
}
else
    require_once('phar://Scripts/PHPExcel.phar/PHPExcel/IOFactory.php');
Cygwin install

Since I’m doing the new install thing, I figured I’d record some of my setup. So here is my cygwin install.

  • Internet utilities: wget, curl, ping, openssh, openssl-devel, mysql, nc
  • Program compilation stuff: gcc-g++, autoconf, pkg-config, automake
  • Programming languages: perl, python
  • Other utilities for programming: git, sqlite3
  • Text editor: nano

Mintty [non-default] options:
  • Text->Locale=en_US UTF-8
  • Mouse->Copy on select=Off
  • Mouse->Clicks place command line cursor=On

I also haven’t decided yet if I will do apache or PHP via cygwin or windows installs yet. May do a post about that later.

Cygwin SIGINT fix for golang

Cygwin has had a long time problem that, depending on your configuration, may cause you to be unable to send a SIGINT (interrupt signal via Ctrl+C) to a native Windows command line executables. As a matter of fact, trying to do so may completely freeze up the console, requiring a process kill of the actual console, bash, and the executable you ran. This problem can crop up for many reasons including the version of Cygwin you are running and your terminal emulator. I specifically installed mintty as my default Cygwin console to get rid of this problem a long time ago (among many other features it had), and now it even has this problem.

While my normal solution is to try and steer clear of native Windows command line executables in Cygwin, this is not always an option. Golang was also causing me this problem every time I ran a network server, which was especially problematic as I would have to ALSO manually kill the server process or it would continue to hold the network port so another test of the code could not use it. An example piece of code is as follows:

package main
import ( "net/http"; "fmt" )
func main() {
	var HR HandleRequest
	if err := http.ListenAndServe("127.0.0.1:81", HR); err!=nil {
		fmt.Println("Error starting server") }
}

//Handle a server request
type HandleRequest struct{}
func (HR HandleRequest) ServeHTTP(w http.ResponseWriter, req *http.Request) {
	fmt.Printf("Received connection from: %s\n", req.RemoteAddr)
}
---
go run example.go

The first solution I found to this problem, which is by far the best solution, was to build the executable and then run it, instead of just running it straight from go.
go build example.go && example.exe
However, as of this post, it seems to no longer work! The last time I tested it and confirmed it was working was about 3 months ago, so who knows what has changed since then.

The second solution is to just build in some method of killing the process that uses “os.Exit”. For example, the following will exit if the user types “exit”
func ListenForExitCommand() {
	for s:=""; s!="exit"; { //Listen for the user to type exit
		if _, err:=fmt.Scanln(&s); err!=nil {
			if err.Error()!="unexpected newline" {
				fmt.Println(err) }
		} else if s=="flush" || s=="exit" {
			//Clean up everything here
		}
	}
	fmt.Println("Exit received, closing process")
	os.Exit(1)
}
and then add the following at the top of the main function:
go ListenForExitCommand() //Listen for "exit" command
Windows “ln” (symbolic linking) support for Cygwin
Cygwin requires lots of tweaks to really be usable
The “ln -s” command in Cygwin creates a fake symbolic link only supported in Cygwin. I whipped up the following script to create a true windows symbolic link that is supported in both Windows and Cygwin (it shows up as a symlink in Cygwin).
TARGET=`echo $1 | perl -pe 's/\\//\\\\/g'`; #Determine the target from the first parameter (where we are linking to). Change forward slashes to back slashes
LINK=`echo $2 | perl -pe 's/\\//\\\\/g'` #Determine the link name from the second parameter (where the symlink is made). Change forward slashes to back slashes
cmd /c mklink $3 $4 $5 $6 "$LINK" "$TARGET" #Perform the windows mklink command with optional extra parameters
Note that for the link name, you have to include the full filename, not just specify a directory.
See here for information on mklink and its switches. Specifically:
  • /d : directory symlink (more hard)
  • /j : directory junction (more soft)
This handles the problem a little more directly than my other post on the topic ("Symlinks in a Windows programming environment").

[Edit on 2016-01-12 @ 12:34am]
And once again, I have a new version of the code. This version has the following advantages:
  • No longer limited to just 4 extra parameters (dynamic instead of static)
  • Can now just specify your directory as the link location, and the filename will be automatically filled in
  • Handles spaces better
Do note, if you want to link directly to a hard drive letter, you must use "c:/" instead of "/cygdrive/c/"

#Get the target and link
TARGET="$1"
shift
LINK="$1"
shift

#If the link is already a directory, append the filename to the end of it
if [ -d "$LINK" ]; then
    #Get the file/directory name without the rest of the path
    ELEMENT_NAME=`echo "$TARGET" | perl -pe 's/^.*?\/([^\/]+)\/?$/$1/'`

    #Append the file name to the target, making sure there is only 1 separating "/"
    LINK=`echo "$LINK" | perl -pe 's/^(.*?)\/?$/$1/'`
    LINK="$LINK"/"$ELEMENT_NAME"
fi

#Replace forward slashes with back slashes
TARGET=`echo $TARGET | perl -pe 's/\\//\\\\/g'`
LINK=`echo $LINK | perl -pe 's/\\//\\\\/g'`

#Perform the windows mklink command with optional extra parameters
cmd /c mklink "$@" "$LINK" "$TARGET"
Symlinks in a Windows programming environment
Windows will get it right one day

I have been having some problems regarding symlinks (symbolic links) for a project that I’ve been working on recently which is requiring work in at least 5 very different operating systems (and about a dozen programming languages). Not many programs support symlinks properly that I have the need to because support for it wasn’t added for NTFS until Windows Vista, and it still has some problems.

It is really great that Windows Vista and Windows 7 now support native symlinks so they can be utilized by programs out of the box. For example, one such instance where I need this a lot is directory relinking in Apache. While Apache’s mod_alias can duplicate the functionality of symlinks for many needs, creating special cases for this one piece of software when distributing a code repository is just not practical, and having proper symlinks natively followed without the program knowing they aren’t the actual file/directory is really the best solution so everything works without special cases.

The way to create NTFS symlinks in Windows Vista+ is through the “mklink” command, which is unfortunately implemented directly in the Window’s command shell, and not a separate executable, so it is not accessible to Cygwin. Further, Cygwin has made a stance to only support reading NTFS symlinks, and not creating them, because they can only be created by administrators, and require specification as to whether the link’s target is a directory or file. Cygwin itself in Windows has had support for symlinks for a long time, but these are not compatible with any program run outside of the Cygwin environment.

Now, my real problem started occurring when trying to use these NTFS symlinks with GIT. While GIT natively supports symlinks, TortoiseGIT doesn’t really support them at all, and throws errors when they are encountered. This is still a big problem that I am going to have to think about :-\. Fortunately, when working with GIT in Cygwin they still work, with caveats. As previously mentioned, only reading the NTFS symlinks in Cygwin work, so when you fetch/pull from a repository and it creates Cygwin style symlinks, Windows still does not read them properly. The following is a script I wrote to change the Cygwin style symlinks into NTFS style symlinks. It can be run from the root folder of the GIT project.

#!/bin/bash
IFS=$'\n' #Spaces do not count as new delimiters

function makewinlink
{
	LINK=$1
	OPTIONS=$2
	TARGET=`find $LINK -maxdepth 0 -printf %l`
	LASTMODTIME=`find $LINK -maxdepth 0 -printf "%t"`
	LINKDIR=`find $LINK -maxdepth 0 -printf %h`
	TARGET=`echo $LINKDIR/$TARGET`
	rm -f $LINK
	cmd /c mklink $OPTIONS "$(cygpath -wa $LINK)" "$(cygpath -wa $TARGET)"
	touch -h -d $LASTMODTIME $LINK
}

#Relink all directories
FILES=`find -type l -print0 | xargs -0 -i find -L {} -type d -maxdepth 0`
for f in $FILES
do
	makewinlink $f /D
done

#Relink all files
FILES=`find -type l -print0 | xargs -0 -i find -L {} -type f -maxdepth 0`
for f in $FILES
do
	makewinlink $f
done

Make sure when committing symlinks in a GIT repository in Windows to use Cygwin with Cygwin style symlinks instead of TortoiseGIT. Also, as previously mentioned, after running this script, TortoiseGIT will show these symlinks as modified :-\. If this is a problem, you can always reverse the process in Cygwin by changing the “cmd /c mklink $OPTIONS” line to a “ln -s” in the above script (note that “target” and “symlink’s name” need to be switched) along with a few other changes.


[EDIT ON 2011-01-03 @ 6:30am] See here for a better example of symlinking in Windows that uses relative paths. [/EDIT]