I had some fun getting OpenCV with CUDA support and the demo to work that required OpenGL that for some yet unknown reason would not connect in properly. So I thought I would share my experience.

First of compile OpenCV with CUDA

You can also enable OpenGL, but this didn’t work for me yet it did enable the #define HAVE_OPENGL compile it up and copy out the brox example and make the following corrections, aka #defeine out the OPENGL window parts.

 1 
 2         cout << "OpenCV / NVIDIA Computer Vision" << endl;
 3         cout << "Optical Flow Demo: Frame Interpolation" << endl;
 4         cout << "=========================================" << endl;
 5 
 6         namedWindow("Forward flow");
 7         namedWindow("Backward flow");
 8 
 9 #ifdef HAVE_OPENGL
10 
11         namedWindow("Needle Map", WINDOW_OPENGL);
12         setGlDevice();
13 
14 #endif
15 
16         namedWindow("Interpolated frame");
17 
18 

To note you may need to #undef HAVE_OPENGL. For a full easy run source you can get from here

Reference: Body of code taken from OpenCV Sample

Left – Forward Optical flow, Right – Backward Optical flow

Some interesting performance info for computation of Brox Optical Flow it took 0.01s, but the Image Copy to GPU took 6.485s [e.g. GpuMat d_frame0(frame0Gray) ]. This was on a laptop spec of i7 (8 thread), 10gb ram, GeForce GT540M 1GB. I will try on another machine sometime see if can get a faster performance specifically in image copy.

Note:

• There may be a better way to overcome the OpenGL requirement this was just a quick work around
• NVIDIA CUDA compiler does not currently support Visual Studio 2012 therefore for CUDA you need to use 2010.
</ul>
Stuart James  ...

After helping as a “Student Helper” (aka Helen’s Minion) for this week at BMVC 2012, I thought would post up a few shots taken over the week. We had a photographer there so haven't taken too many but to give you a taster of the conference.

Conference Reception Inside Guildford Cathedral

BMVC Helper T-Shirt Back, Surrey last hosted in ‘93!

Our gracious hosts minus Krystian welcoming us to Brooklands Museum

Looking around the Brooklands

…

Engine room with moving parts!

Conference Banquet

Andrew Fitzgibbon introducing John Illingworth as BMVA Distinguished Fellow

Finally get ready for next year, BMVC 2013 hosted by University of Bristol

Stuart James  …

In many scenarios knowing how an object moves between frames is important, for example if you are trying to separate moving objects from the background or to calculate the speed of a tennis ball. Generally within Computer Vision people default to the Lucas Kanade[1] Tracker or the Pyramid expansion, although this tracker is commonly used it is not generally considered to work particularly well in real world scenarios. To help understand a little better I have compared 3 different techniques over a selection of different video types varying in object and quality.

We compare:

• Lucas-Kanade
• Brox[2]
• Sun[3]
</ul>

To note in these experiments we use default settings provided by the implementations, with tweaking you will get better results. Also I am sure there are faster implementations than the matlab ones used. I also realise they are very hard test cases with low quality, lots of noise motion blur.

Vector Comparison

Frame 1	Frame 2	LucasKanade	Brox	Sun

 

Performance

	Lucas-Kanade	Brox	Sun
Horse (1280x720)	105s	64s	619s
Dance(400x400)	26s	15s	134s
Snowboarder(720x576)	30s	28s	294s

 

Conclusion

Well as you can see they are inconclusive results, Sun’s approach I would say works better at too higher computational cost LK works well on the dance footage or at least in that specific example, less noisy frames of the dance footage I have found LK performed poorly on. Brox, seems generally not to cope very well in these cases. To conclude, optical flow is still a very very incomplete field, you have to test on your problem to find the best solution.

References

[1] Lucas, B., & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. Proceedings of the 7th international joint … (Vol. 130, pp. 121–129). Retrieved from http://www.ri.cmu.edu/pub_files/pub3/lucas_bruce_d_1981_1/lucas_bruce_d_1981_1.ps.gz

[2] Brox, T., Bruhn, A., Papenberg, N., & Weickert, J. (2004). High accuracy optical flow estimation based on a theory for warping. Computer Vision-ECCV 2004, 4(May), 25–36. Retrieved from http://www.springerlink.com/index/87a4ckjqm92lp3j9.pdf

[3] Sun, D., Roth, S., & Black, M. J. (2010). Secrets of optical flow estimation and their principles. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2432–2439. doi:10.1109/CVPR.2010.5539939

Disclaimer

This evaluation was done by taking openly available code for full references see the download file, that contains readme references in the relevant subfolders. If you have developed your own implementation of one of these methods or an alternative approach it would be great to expand with your results.


Stuart James  ...

Often API documentation can be difficult to follow and your memory might fail for algorithms you learnt in school, so a simple example often explains how to use a function. Well unless you are very familiar with the way Boost Graph library works then you may find the Dijkstra shortest path documentation confusing.

So this is what you get from boost…

 1 typedef adjacency_list < listS, vecS, directedS,
 2   no_property, property < edge_weight_t, int > > graph_t;
 3 typedef graph_traits < graph_t >::vertex_descriptor vertex_descriptor;
 4 typedef graph_traits < graph_t >::edge_descriptor edge_descriptor;
 5 typedef std::pair<int, int> Edge;
 6 
 7 const int num_nodes = 5;
 8 enum nodes { A, B, C, D, E };
 9 char name[] = "ABCDE";
10 Edge edge_array[] = { Edge(A, C), Edge(B, B), Edge(B, D), Edge(B, E),
11   Edge(C, B), Edge(C, D), Edge(D, E), Edge(E, A), Edge(E, B)
12 };
13 int weights[] = { 1, 2, 1, 2, 7, 3, 1, 1, 1 };
14 int num_arcs = sizeof(edge_array) / sizeof(Edge);
15 
16 graph_t g(edge_array, edge_array + num_arcs, weights, num_nodes);
17 property_map<graph_t, edge_weight_t>::type weightmap = get(edge_weight, g);
18 
19 std::vector<vertex_descriptor> p(num_vertices(g));
20 std::vector<int> d(num_vertices(g));
21 vertex_descriptor s = vertex(A, g);
22 
23 dijkstra_shortest_paths(g, s, predecessor_map(&p[0]).distance_map(&d[0]));

Taken from http://www.boost.org/doc/libs/1_49_0/libs/graph/example/dijkstra-example.cpp

From this example you are able to get the predecessor map and the distances travelled from a root node to all other nodes. But this doesn’t answer the question of how you get the shortest path from point A to point B. The way to do this is called backtracking and is very easy but not necessarily obvious when you don't know the data structures.

1 // Backtrack from end node, storing distances
2 int target = vertex(B, g);
3 std::vector<int> nodes;
4 std::vector<float> distance;
5 do{
6     nodes.push_back(target);
7     distances.push_back(d[target]);
8     target = p[target];
9 }while(target != s);

The vector nodes then contains a list of nodes passed through, that can be reversed easily using std lib functions to find the route from A to B.

Stuart James  …

Played my first game of Carcassonne tonight, was interesting didn’t really construct into anything nice though that I am disappointed by! Would of been nice to have a good map of castles and paths. May have more luck next time.

Stuart James  …