[Input] Target image image.bmp
Here, we introduce how to implement Lazy Snapping using OpenCV 2.4.11. OpenCV's grabCut function is used. GUI is not implemented here. The strokes which represents the foreground or the background is assumed to be given.
// Include directory C:\OpenCV2.4.11\build\include
// Library directory (x86)
C:\OpenCV2.4.11\build\x86\vc12\lib
// Library directory (x64)
C:\OpenCV2.4.11\build\x64\vc12\lib
// PATH (x86) C:\OpenCV2.4.11\build\x86\vc12\bin
// PATH (x64) C:\OpenCV2.4.11\build\x64\vc12\bin
#if _DEBUG
#pragma comment(lib,
"opencv_core2411d.lib")
#pragma comment(lib,
"opencv_highgui2411d.lib")
#pragma comment(lib,
"opencv_imgproc2411d.lib")
#else
#pragma comment(lib,
"opencv_core2411.lib")
#pragma comment(lib,
"opencv_highgui2411.lib")
#pragma comment(lib,
"opencv_imgproc2411.lib")
#endif
#include
<opencv2/opencv.hpp>
int main()
{
// Input
cv::Mat
inputImage = cv::imread("image.bmp", 1);
cv::Mat
inputMask = cv::imread("mask.bmp", 1);
if
(inputImage.data == NULL || inputMask.data ==
NULL) return
1;
if
(inputImage.size() !=
inputMask.size())
return 1;
cv::Mat
strokeImage = inputImage.clone();
cv::Mat
processMask = cv::Mat(inputMask.rows,
inputMask.cols, CV_8UC1);
cv::Mat
regionImage = inputMask.clone();
cv::Mat
compositeImage = inputImage.clone();
cv::Mat
fgImage = inputImage.clone();
// Mask
for (int
y = 0; y < inputMask.rows; y++) {
for
(int x = 0; x < inputMask.cols; x++) {
cv::Vec<unsigned
char, 3> color = inputMask.at<cv::Vec<unsigned
char, 3>>(y, x);
if (color[0]
== 0 && color[1]
== 0 && color[2]
== 255) {
processMask.at<unsigned char>(y, x) =
cv::GC_FGD;
strokeImage.at<cv::Vec<unsigned
char, 3>>(y, x)[0]
= 0;
strokeImage.at<cv::Vec<unsigned
char, 3>>(y, x)[1]
= 0;
strokeImage.at<cv::Vec<unsigned
char, 3>>(y, x)[2]
= 255;
}
else if (color[0]
== 255 && color[1]
== 0 && color[2]
== 0) {
processMask.at<unsigned char>(y, x) =
cv::GC_BGD;
strokeImage.at<cv::Vec<unsigned
char, 3>>(y, x)[0]
= 255;
strokeImage.at<cv::Vec<unsigned
char, 3>>(y, x)[1]
= 0;
strokeImage.at<cv::Vec<unsigned
char, 3>>(y, x)[2]
= 0;
}
else {
processMask.at<unsigned char>(y, x) = (x + y) %
2 == 0 ? cv::GC_PR_FGD : cv::GC_PR_BGD;
}
}
}
// LazySnapping
cv::Mat bgdModel;
cv::Mat fgdModel;
cv::Rect rect;
const int
iterCount = 4;
cv::grabCut(inputImage, processMask, rect,
bgdModel, fgdModel, iterCount, cv::GC_INIT_WITH_MASK);
// GrabCut
// rect = cv::Rect(13, 32, 302, 155);
// cv::grabCut(inputImage, processMask, rect,
bgdModel, fgdModel, iterCount, cv::GC_INIT_WITH_RECT);
// Foreground
for (int
y = 0; y < inputMask.rows; y++) {
for
(int x = 0; x < inputMask.cols; x++) {
compositeImage.at<cv::Vec<unsigned
char, 3>>(y, x)[0]
/= 2;
compositeImage.at<cv::Vec<unsigned
char, 3>>(y, x)[1]
/= 2;
compositeImage.at<cv::Vec<unsigned
char, 3>>(y, x)[2]
/= 2;
int category = processMask.at<unsigned
char>(y, x);
if (category == cv::GC_FGD || category ==
cv::GC_PR_FGD) {
regionImage.at<cv::Vec<unsigned
char, 3>>(y, x)[0]
= 0;
regionImage.at<cv::Vec<unsigned
char, 3>>(y, x)[1]
= 0;
regionImage.at<cv::Vec<unsigned
char, 3>>(y, x)[2]
= (category == cv::GC_FGD ? 255 : 127);
compositeImage.at<cv::Vec<unsigned
char, 3>>(y, x)[0]
+= 0;
compositeImage.at<cv::Vec<unsigned
char, 3>>(y, x)[1]
+= 0;
compositeImage.at<cv::Vec<unsigned
char, 3>>(y, x)[2]
+= 128;
}
else if (category == cv::GC_BGD || category ==
cv::GC_PR_BGD) {
regionImage.at<cv::Vec<unsigned
char, 3>>(y, x)[0]
= (category == cv::GC_BGD ? 255 : 127);
regionImage.at<cv::Vec<unsigned
char, 3>>(y, x)[1]
= 0;
regionImage.at<cv::Vec<unsigned
char, 3>>(y, x)[2]
= 0;
compositeImage.at<cv::Vec<unsigned
char, 3>>(y, x)[0]
+= 128;
compositeImage.at<cv::Vec<unsigned
char, 3>>(y, x)[1]
+= 0;
compositeImage.at<cv::Vec<unsigned
char, 3>>(y, x)[2]
+= 0;
fgImage.at<cv::Vec<unsigned
char, 3>>(y, x)[0]
= 255;
fgImage.at<cv::Vec<unsigned
char, 3>>(y, x)[1]
= 255;
fgImage.at<cv::Vec<unsigned
char, 3>>(y, x)[2]
= 255;
}
}
}
// Output
cv::imwrite("stroke.bmp",
strokeImage);
cv::imwrite("region.bmp",
regionImage);
cv::imwrite("composite.bmp",
compositeImage);
cv::imwrite("foreground.bmp",
fgImage);
return 0;
}
[Input] Target image image.bmp
[Input] User stroke (Red: foregound, blue: background) mask.bmp
Mixture of the target image and the strokes (This image is for visualization
purpose) stroke.bmp
[Output] Region segmentation result (Red: foreground, blue: background) region.bmp
[Output] Mixture of the target image and the segmentation result composite.bmp
[Ouput] Target image with its pixel which is judged as background filled with
white color foreground.bmp
This page explains the example of using OpenCV's grabCut function. Explanation of GrabCut and Lazy Snapping is omitted. If you want to know how to use OpenCV's grabCut function as GrabCut, please read other webpages. This page explains how to use OpenCV's grabCut function as Lazy Snapping. grabCut function is as follows.
void cv::grabCut( InputArray _img, InputOutputArray _mask, Rect rect, InputOutputArray _bgdModel, InputOutputArray _fgdModel, int iterCount, int mode )
Set the target input image to _img. This variable is 3-channel 8-bit cv::Mat
type variable.
_mask is explained later.
As for
rect, just use cv::Rect variable when you use as Lazy Snapping.
As for
_bgdModel and _fgdModel, just set cv::Mat variable.
iterCount is the number of iteration. Set a value, for example, from 1 to 10.
As for
mode, set cv::GC_INIT_WITH_MASK when you use as Lazy Snapping.
_mask is a 1-channel 8-bit cv::Mat variable. Size is the same as the target
input image. For each pixel, the value of one of the followings are set.
cv::GC_BGD // This value is 0. The user has set this pixel as background.
cv::GC_FGD // This value is 1. The user has set this pixel as foreground.
cv::GC_PR_BGD // This value is 2. This pixel is possibly background.
cv::GC_PR_FGD // This value is 3. This pixel is possibly foreground.
The pixel which is set as GC_BGD or GC_FGD would not change during the further computation. grabCut function adequately assigns either GC_PR_BGD or GC_PR_FGD for the pixels where GC_PR_BGD or GC_PR_FGD is set beforehand.
Be careful not to use _mask which is initialized by zero. 0 is GC_BGD, so such pixel remain unchanged when grabCut function is called. If you want to let grabCut function compute whether the pixel is foreground or background, you have to set GC_PR_BGD or GC_PR_FGD.
Before calling grabCut function, set initival value for each pixel of _mask. You have to set either GC_PR_BGD or GC_PR_FGD for the pixel except for GC_BGD or GC_FGD. Set GC_PR_BGD if the probability to be background is high, ore set GC_PR_FGD if the probability to be foregound is high.
The region segmentation result is overwritten to _mask after calling grabCut function. The pixel where GC_BGD or GC_FGD is set remains unchanged. Other pixels are GC_PR_BGD or GC_PR_FGD. The pixel which is GC_FGD or GC_PR_FGD is foreground. The pixel which is GC_BGD or GC_PR_BGD is backround. This is the region segmentation result.
