Putting CG shaders where VS can find them

For a while, I was having problems running my CG tests out of visual studio. It couldn't find my shader files, so I had to run from the command line. Not only did this add some steps, it meant I couldn't debug.

Turns out, all I had to do was move the shader to where VS creates new classes by default (another level deeper in the directory than where I normally put my source code). I went ahead and moved all my code and shaders there, and now it will happily find the shaders when I run via F5.

Handy Visual Studio shortcuts

When I discover something nifty in Visual Studio, I'll post it here:

Shortcut	Name	Comment
Ctrl + .	Intelisense	If you've typed enough of a word that there's only one completion available, it'll finish it for you (or bring up a list of other possible completions)
Ctrl + ,	Navigate To	Search project (solution?) for member and function definitions

For now, more goodness here

"Show All Files" crash work-around (Visual Studio 2010)

When I upgraded to Visual Studio 2010, I hit the following problem: after converting my old project to the new format, I made the mistake of clicking on "Show All Files". Once you've done this, it seems you can't go back -- any time I tried to deselect it, VS crashed.

Here's how I fixed it:

1. Open up PowerShell in your project folder


2. Run ls -rec | select-string Show


3. Use your favorite text editor to set all references to "ShowAllFiles" to false

In my case, this ended up being [ProjectName].vcxproj.user, but do the Select-String search to be sure.

The "Uncanny valley" of automation

Automation is not the panacea I thought it was.

Implemented incorrectly, an automation system will waste more of your time than it saves. Sure, you won't have to run through the same checklists over and over, but your time will instead be spent tracking down a disproportionately large number of false positives. At first, I thought: "Hey, at least it's code! It's got to be more fun than just using the product, right?" Now I'm not so sure.

In robotics and computer graphics, there's a problem known as the "uncanny valley": when you get closer to modeling realistic human behavior, the results can be unsettling. I propose that there's a similar problem with automation: the closer you get to trying to simulate human behavior, the more problematic the results. (A bit of a stretch, and due to an entirely different set of problems, but bear with me).

Humans are good at dealing with the abstract. Take, for instance, the hard to read text you're supposed to identify when signing up for something (it's called a "CAPTCHA"). Reading these is (usually) easy for a human; we look at the blurry distorted mess and see letters. This is something we are good at. Writing a program that can read those things, on the other hand, is a difficult computer science problem.

Conversely, say you had to create dozens of accounts somewhere (perhaps as part of testing an online service). Completely filling out the name, address, interests, secret question, etc, is boring, repetitive, and prone to mistakes. These sort of tasks are not our forte. However, it's easy to write a script that fills in all of the fields for you. (Especially with a web page, since you can interact directly with the DOM).

This example illustrates the balance between manual and automatic tasks. Humans are good at abstract reasoning and big picture. Computers are good at repetition and precision (note I didn't say accuracy :-P)

Human users have no problem with slight changes in design or layout (and sometimes they won't even notice); computers tend to go apeshit. This is what causes so many of the automation failures you'll be tracking down. Maybe the browser started minimized, or another window popped up in front of it and stole focus (automatic updates, anyone?). Maybe the page took a few seconds longer to load, and the expected items weren't there when the script checked for them. Maybe the designer moved a button or changed a label. Or, maybe your test itself was wrong! There may be a legitimate outcome that you, author of the test, didn't think of. (When tests are code, they can have bugs too!)

Where computers shine are precisely defined problems. This suits them well for testing behind the scenes, where the input and output is not so abstract. If I send this packet, does the server give back that response? If I call this function with this value, do I get back that answer? What makes this sort of testing and verification so boring (and difficult) for humans is precisely what makes it so perfect for computers.

Another area where automation shines is tools. Running a series of installers and patches...opening up dozens of web browsers to a series of long, convoluted URLs...finding all the logs from a certain period of time, spread over dozens of computers, compressing them, copying them to a central repository, and emailing all interested parties about their availability...these are the sort of mindless, error prone tasks that waste tester time and are just begging to be automated.

The take away from this is that automated tests are most useful when they augment human testing, not try to replace it. Automation should simplify the lives of human testers -- by taking over the tasks humans are inherently bad at -- so they can focus on what they do well: finding problems in the user experience.

Compile Cg shaders from command line

Use cgc to compile Cg shader programs from the command line. (cgc --help for usage). Useful for seeing errors/warnings and instruction counts.

Cheat sheet:
Vertex shader: cgc vert.cg -entry main -profile arbvp1
Fragment shader: cgc frag.cg -entry main -profile arbfp1

Rick Barraza Silverlight blog link

This looks like an interesting source of Silverlight visual effects. Check back here later: http://cynergysystems.com/blogs/page/rickbarraza

Authentication for Coded VSTS Web Tests

To set credentials for a Visual Studio WebTest programatically, simply add the following to the class that extends WebTest:

this.UserName = "domain\\user"
this.Password = "pass"


Easily run commands on many machines with powershell

Need to run a series of commands on a bunch of machines in a row? Try something like this in PowerShell:

01,02,03,04,05,08,09,10,19,20,21 |
%{"base10name12{0:D2}" -f $_} | %{
Enable-WSManCredSSP client $_
invoke-command -comp $_ { cp D:\logs\* \\server\share\logs }
restart-computer $_
}


Rudimentary PowerShell wrapper (CMD.exe replacement)

PowerShell rocks, but CMD.exe sucks is too limited for my tastes. I'm going to try making a wrapper/CMD.exe replacement for PowerShell. And what better language to write it in than PowerShell script itself? I'm calling it "IggyPosh" for now (POwer SHell).

Here's an early draft. No prompt or command history, but you can type commands and they are executed, so it's a start.


function test($str) {
$str = $rs.CreatePipeline($str).Invoke() | Out-String
$history.Text += $str -replace "\s+$([Environment]::NewLine)",[Environment]::NewLine
$history.SelectionStart = $history.Text.Length
$history.ScrollToCaret()
}

# Prepare the window
[Reflection.Assembly]::LoadWithPartialName("System.Windows.Forms")
$form = New-Object Windows.Forms.Form
$form.Text = "IggyPosh"
$form.Size = New-Object Drawing.Point 600,400

$history = New-Object Windows.Forms.TextBox
$history.Multiline = $true
$history.ReadOnly = $true
$history.ScrollBars = [System.Windows.Forms.ScrollBars]::Vertical
$history.Dock = [System.Windows.Forms.DockStyle]::Fill
$history.BackColor = [System.Drawing.Color]::FromArgb(0,64,0)
$history.ForeColor = [System.Drawing.Color]::FromArgb(224,224,224)
$history.Font = New-Object System.Drawing.Font("Consolas", 9)
$form.Controls.Add($history)

$command = New-Object Windows.Forms.TextBox
$command.Dock = [System.Windows.Forms.DockStyle]::Bottom
$command.add_KeyDown({
if ($_.KeyCode -eq "Enter") { test($command.Text); $command.Text="" }
})
$form.Controls.Add($command)

# Try to init a powershell instance we can talk to
$rs = [System.Management.Automation.Runspaces.RunspaceFactory]::CreateRunspace()
$rs.Open()

$form.ShowDialog()


Create directories and upload files to Sharepoint from PowerShell

Create a new folder on Sharepoint server hosted at http://server/:
$url = "http://server/Shared%20Documents/newfolder/"
$req = [System.Net.HttpWebRequest]::Create($url)
$req.Credentials = [System.Net.CredentialCache]::DefaultCredentials
$req.Method = "MKCOL"
$res = $req.GetResponse()


Add a file to that folder:
$dest = $url+"newfile.txt"
$src = "C:\myfile.txt"
$wc = New-Object System.Net.WebClient
$wc.Credentials = [System.Net.CredentialCache]::DefaultCredentials
$wc.uploadfile("$dest", "PUT", $src)


IEEE float special values (infinity, NaN)

I couldn't find anything in the Cg documentation on how to specify infinity or NaN, but these seem to work:

Infinity: 0x7f800000
-Infinity: 0xff800000
NaN: 0x7fc00000

PowerShell script to play latest video linked on web page

My auto-piracy machine do-it-yourself media center lists the most recently downloaded TV shows in a way that looks like this (in HTML, that is):

<a href="http://www.blogger.com/tv/How%20I%20Met%20Your%20Mother/05/how.i.met.your.mother.s05e11.720p.hdtv.x264-ctu.mkv">Season 5 Episode 11</a>;

The web interface isn't bad, but I like to mess around with scripting and command line stuff, so here's a powershell script that will play the latest show matching a given string. (Examples: latest how, latest -n 2 dollhouse)


param(
[switch]$switch = $false,
[string]$query = $(Read-Host -prompt "Keyword"),
[int]$num = $(if ($switch) {Read-Host -prompt "Number"} else {1})
)

$player = "C:\Program Files\SMPlayer\smplayer.exe"
$client = New-Object System.Net.WebClient
$data = $client.DownloadData("http://spot/")
$page = [System.Text.Encoding]::ASCII.GetString($data)

# They are displayed on the page with newest on top, but we want to play them in order (oldest to newest)
$matches = [regex]::Matches($page, "href=`"([^\<\>]*?$query[^\<\>]*?\.[^\<\>]*?)`"", "IgnoreCase") | Select-Object -first $num | Sort-Object

$files = ""
foreach ($m in $matches) {
$files += "'http://spot/$($m.groups[1])' "
}

if ($files.Length -gt 0) {
echo $files
Start-Process $player $files
}
else { echo "Could not find anything matching '$query'" }


Notes:

• [^\<\>] means "any character except < or >". The *? means "any number of the preceding". So in this case, it means "any number of characters except < or >". Useful for reading the text between tags.


• The bit at the top lets you optionally specify the number of matches to use with -num (or -n), defaulting to 1 otherwise. If you don't give it a query, it will prompt you. (It doesn't properly handle the case of -num w/o a number, but I don't care at this point. You'll get an error message, just not a nice one).

Notes-to-self: actually applying the sine curve stuff

A practical variation of the previous idea:

Use one quarter of the sine wave (covex up, convex down, concave down, concave up) for each block. Every other height will be used as an actual height, and the heights in-between will be used to determine whether the segment is convex or concave.

Web-scraping web comics

A template for web-scraping comics, using Piled Higher & Deeper as an example (in Python; if on windows, you'll need wget from Cygwin):

import urllib2
import re
import os
import sys

url = "http://www.phdcomics.com/comics/archive.php?comicid="

for id in range(int(sys.argv[1]), int(sys.argv[2])):
print `id`+"..."
try:
page = urllib2.urlopen(url+`id`).read()
img = re.search('img src=(http://www.phdcomics.com/comics/archive/.+?.gif)', page).group(1)
os.system('wget -nc '+img)
except urllib2.HTTPError:
print "No page_id " + `id`


Notes-to-self: sinusoidal curve fitting

One ppt
Another ppt

y = C + Asin(Bx - D)

A = (largest-smallest)/2 (determine the amplitude)
C = (largest+smallest)/2 (determine the vertical shift)
B = 2pi/P (P = period? delta time/distance?) (determine period?)
D = "must use A, B, C and a point (x,y)" (determine horizontal shift)

Looks like it requires 3 points. Maybe divide height-map into 3x3 "squares"?
- Can't find sin(x)*sin(z) that goes through them all, how to handle that?
- Some points will have to be left out

Maybe do it like this?
0 z 0
x x/z x
0 z 0
- Results in about half the points being left out...much of a problem? (increase height-map resolution)

Notes-to-self: next steps for sine-based terrain

(notes to self; not intended to be interesting/meaningful/relevant to anyone else)

So I have terrain based on sin(x)*sin(z) working pretty well (where "pretty well" is defined as 40-60+ fps on my iMac...the shader is too complicated to run on my laptop's integrated intel card, but more on that later...maybe). It would look alright if used for, say, an open stretch of ocean, but it's way too boring for something that's supposed to be actual land. So...what's next?

Well, first, some irritations with my current method. Chief among them is that I cannot change the landscape without changing both the code and the shader (the code sets up the blocks the shader carves away from). The main reason for this is that the formula (sin(x)*sin(z)) is hard-coded into the shader. Ideally, I could change just change code. (Actually, ideally I wouldn't have to change the code either; it would read the formula from a text file and everything else would be magically figured out.) The way I see to do the first part: generalize the formula in the shader so that it accepts parameters from the code (ie: a*sin(bx + c) + d).

So here's a thought for "what's next", which attempts to address several other problems I'm having at the moment: code hands block to shader, with 8 parameters. The shader will evaluate a*sin(bx+c)+d * e*sin(fz+g)+h on the block, in modelspace. This will simplify the shader, as it will not need to know anything about where in the world the object is, and make it more flexible (we just want the shader to make the GPU do math work for us; we don't want to simulate state with it).

The simplest use of this is to easily tweak the whole scene's frequency, amplitude, etc. from code. However, another possible (ab)use is stringing together blocks of different frequency or amplitude together to make a more interesting level. One problem that immediately comes to mind is differentiability: if I'm just stringing sections of sine waves together, they certainly won't be differentiable...but the question is, how bad will it look? Given reasonable constraints, will it look "good enough"?

Keeping the current semi-goal of smooth curvy terrain from a height map in mind: could I take a "square" of the height map, look at the differences, and find a chunk of sin(x)*sin(z) that, with appropriate a-h parameters as described above, approximates it? It seems feasible enough (in my tired state) to warrant further investigation.

Cg syntax highlighting in vim

1. Download cg.vim to ~/.vim/syntax/. (more info here. Note: the file on that page is DOS formatted; my link is to a converted version)


2. Enable automatic highlighting by adding this to ~/.vimrc:
   au BufNewFile,BufRead *.cg setf cg

Easy iPhone landscape orientation

Put one of the following in your AppDelegate's applicationDidFinishLaunching method:

• application.statusBarOrientation = UIInterfaceOrientationLandscapeRight;
• application.statusBarOrientation = UIInterfaceOrientationLandscapeLeft;

This has several benefits:

• The status bar, keyboard, and alerts are oriented properly
• The iPhone simulator will orient itself properly

Matrix transformations and gluLookAt in Cg

OpenGL has some nice helper functions for matrix manipulation: glTranslate, glRotate, and glScale.

GlRotate is especially involved, so I made a version in Cg (based on the matrix in the spec):

float4x4 getRotateMatrix(float theta, float3 axis) {
float x = axis.x, y = axis.y, z = axis.z, c = cos(theta), s = sin(theta);
float4x4 matrix = float4x4(
x*x*(1-c)+c, x*y*(1-c)-z*s, x*z*(1-c)+y*s, 0,
y*x*(1-c)+z*s, y*y*(1-c)+c, y*z*(1-c)-x*s, 0,
x*z*(1-c)-y*s, y*z*(1-c)+x*s, z*z*(1-c)+c, 0,
0, 0, 0, 1
);
return matrix;
}


Another especially useful function is gluLookAt, which lets you specify camera position and tell it to look at a specific point. I downloaded the Mesa3D code and made the following Cg function based on it:

float4x4 getLookAt(float3 eye, float3 center, float3 up) {
float3 forward = normalize(center - eye);
float3 side = normalize(cross(forward, up)); /* Side = forward x up */
up = cross(side, forward); /* Recompute up as: up = side x forward */

return float4x4(
side.x, up.x, -forward.x, -eye.x,
side.y, up.y, -forward.y, -eye.y,
side.z, up.z, -forward.z, -eye.z,
0, 0, 0, 1
);
}


Using opengl, glut, and cg with Visual Studio

These pages describe the process (the first one is out of date, but the changes to directory structure are minor):

OpenGL and GLUT
Cg

In case those pages go away, it boils down to this:

GLUT

• Get GLUT for Win32


• Copy...




• glut32.dll to %WinDir%\System


• glut32.lib to $(VSDir)\VC\lib


• glut.h to $(VSDir)\include\GL




• ...where $(VSDir) is something like C:\Program Files\Microsoft Visual Studio 9.0\VC
  

Cg

• Install the latest Cg Toolkit

Visual Studio

• Make a new Win32 Console Application project: File->New Project->Visual C++->Win32->Win32 Console Application


• In the "Application Settings" page of the wizard, make sure Application type is Console Application and Empty project (under Additional options) is checked.


• Go to Project->(project name) Settings




• Set Configuration to All Configurations


• Expand Configuration Properties


• In C/C++->General, add $(CG_INC_PATH) to Additional Include Directories


• In Linker->General, add $(CG_LIB_PATH) to Additional Library Directories


• In Linker->Input, add opengl32.lib glu32.lib glut32.lib cg.lib cgGL.lib to Additional Dependencies