OpenCV warpperspective

For some reason whenever I use OpenCV's warpPerspective() func开发者_Python百科tion, the final warped image does not contain everything in the original image. The left part of the image seems to get cut off. I think the reason why this is happening is because the warped image is created at the leftmost position of the canvas for the warpPerspective(). Is there some way to fix this? Thanks

---

The problem occurs because the homography maps part of the image to negative x,y values which are outside the image area so cannot be plotted. what we wish to do is to offset the warped output by some number of pixels to 'shunt' the entire warped image into positive coordinates(and hence inside the image area).

Homographies can be combined using matrix multiplication (which is why they are so powerful). If A and B are homographies, then AB represents the homography which applies B first, and then A.

Because of this all we need to do to offset the output is create the homography matrix for a translation by some offset, and then pre-multiply that by our original homography matrix

A 2D homography matrix looks like this :

[R11,R12,T1]
[R21,R22,T2]
[ P , P , 1]

where R represents a rotation matrix, T represents a translation, and P represents a perspective warp. And so a purely translational homography looks like this:

[ 1 , 0 , x_offset]
[ 0 , 1 , y_offset]
[ 0 , 0 ,    1    ]

So just premultiply your homography by a matrix similar to the above, and your output image will be offset.

(Make sure you use matrix multiplication, not element wise multiplication!)

---

Matt's answer is a good start, and he is correct in saying you need to multiply your homography by

[ 1 , 0 , x_offset]
[ 0 , 1 , y_offset]
[ 0 , 0 ,    1    ]

But he does not specify what x_offset and y_offset are. Other answers have said just take the perspective transform, but that is not correct. You want to take the INVERSE perspective transform.

Just because a point 0,0 transforms into, say, -10,-10, does not mean that shifting the image by 10,10 will result in a non-cropped image. This is because point 10,10 does not necessarily map into 0,0.
What you want to do is find out what point would map into 0,0, and shift the image by that much. To do that you take the inverse (cv2.invert) of the homography and apply perspectiveTransform.

does not imply:

You need to apply a reverse transform to find the correct points.

This will get the correct x_offset and y_offset to align your top left point. From there to find the correct bounding box and fit the entire image perfectly, you need to figure out the skew (how much the image slants left or up after your normal, non-inverse, transformation) and add that amount to your x_offset and y_offset as well.

EDIT: This is all theory. Images are a few pixels off in my tests, I'm not sure why.

---

The secret comes in two parts: the transform matrix (homography), and the resulting image size.

• calculate a correct transform by using the getPerspectiveTransform(). Take 4 points from the original image, calculate their correct position in the destination, put them in two vectors in the same order, and use them to compute the perspective transform matrix.

• Make sure the destination image size (third parameter for the warpPerspective()) is exactly what you want. Define it as Size(myWidth, myHeight).

---

I have done one method... It is working.

  perspectiveTransform(obj_corners,scene_corners,H);
int maxCols(0),maxRows(0);

 for(int i=0;i<scene_corners.size();i++)
{
   if(maxRows < scene_corners.at(i).y)
        maxRows = scene_corners.at(i).y;
   if(maxCols < scene_corners.at(i).x)
        maxCols = scene_corners.at(i).x;
}

I just find the maximum of the x points and y points respectively and put it on

warpPerspective( tmp, transformedImage, homography, Size( maxCols, maxRows ) );

---

Try the below homography_warp.

void homography_warp(const cv::Mat& src, const cv::Mat& H, cv::Mat& dst);

src is the source image.

H is your homography.

dst is the warped image.

homography_warp adjust your homography as described by https://stackoverflow.com/users/1060066/matt-freeman in his answer https://stackoverflow.com/a/8229116/15485

// Convert a vector of non-homogeneous 2D points to a vector of homogenehous 2D points.
void to_homogeneous(const std::vector< cv::Point2f >& non_homogeneous, std::vector< cv::Point3f >& homogeneous)
{
    homogeneous.resize(non_homogeneous.size());
    for (size_t i = 0; i < non_homogeneous.size(); i++) {
        homogeneous[i].x = non_homogeneous[i].x;
        homogeneous[i].y = non_homogeneous[i].y;
        homogeneous[i].z = 1.0;
    }
}

// Convert a vector of homogeneous 2D points to a vector of non-homogenehous 2D points.
void from_homogeneous(const std::vector< cv::Point3f >& homogeneous, std::vector< cv::Point2f >& non_homogeneous)
{
    non_homogeneous.resize(homogeneous.size());
    for (size_t i = 0; i < non_homogeneous.size(); i++) {
        non_homogeneous[i].x = homogeneous[i].x / homogeneous[i].z;
        non_homogeneous[i].y = homogeneous[i].y / homogeneous[i].z;
    }
}

// Transform a vector of 2D non-homogeneous points via an homography.
std::vector<cv::Point2f> transform_via_homography(const std::vector<cv::Point2f>& points, const cv::Matx33f& homography)
{
    std::vector<cv::Point3f> ph;
    to_homogeneous(points, ph);
    for (size_t i = 0; i < ph.size(); i++) {
        ph[i] = homography*ph[i];
    }
    std::vector<cv::Point2f> r;
    from_homogeneous(ph, r);
    return r;
}

// Find the bounding box of a vector of 2D non-homogeneous points.
cv::Rect_<float> bounding_box(const std::vector<cv::Point2f>& p)
{
    cv::Rect_<float> r;
    float x_min = std::min_element(p.begin(), p.end(), [](const cv::Point2f& lhs, const cv::Point2f& rhs) {return lhs.x < rhs.x; })->x;
    float x_max = std::max_element(p.begin(), p.end(), [](const cv::Point2f& lhs, const cv::Point2f& rhs) {return lhs.x < rhs.x; })->x;
    float y_min = std::min_element(p.begin(), p.end(), [](const cv::Point2f& lhs, const cv::Point2f& rhs) {return lhs.y < rhs.y; })->y;
    float y_max = std::max_element(p.begin(), p.end(), [](const cv::Point2f& lhs, const cv::Point2f& rhs) {return lhs.y < rhs.y; })->y;
    return cv::Rect_<float>(x_min, y_min, x_max - x_min, y_max - y_min);
}

// Warp the image src into the image dst through the homography H.
// The resulting dst image contains the entire warped image, this
// behaviour is the same of Octave's imperspectivewarp (in the 'image'
// package) behaviour when the argument bbox is equal to 'loose'.
// See http://octave.sourceforge.net/image/function/imperspectivewarp.html
void homography_warp(const cv::Mat& src, const cv::Mat& H, cv::Mat& dst)
{
    std::vector< cv::Point2f > corners;
    corners.push_back(cv::Point2f(0, 0));
    corners.push_back(cv::Point2f(src.cols, 0));
    corners.push_back(cv::Point2f(0, src.rows));
    corners.push_back(cv::Point2f(src.cols, src.rows));

    std::vector< cv::Point2f > projected = transform_via_homography(corners, H);
    cv::Rect_<float> bb = bounding_box(projected);

    cv::Mat_<double> translation = (cv::Mat_<double>(3, 3) << 1, 0, -bb.tl().x, 0, 1, -bb.tl().y, 0, 0, 1);

    cv::warpPerspective(src, dst, translation*H, bb.size());
}

---

warpPerspective() works fine. No need to rewrite it. You probably use it incorrectly.

Remember the following tips:

1. (0,0) pixels is not in the center but rather left-upper corner. So if you magnify the image x2 you will lose the lower and right parts, not the border (like in matlab).
2. If you warp image twice it is better to multiply transformations and activate the function once.
3. I think it works only on char/int matrices and not on float/double.
4. When you have a transformation, first zoom/skew/rotation/perspective are applied and finally the translation. So if part of the image is missing just change the transation (two upper rows of last column) in the matrix.

---

An easy way to fix the issue of the warped image being projected outside the warping output is to translate the warped image to the right position. The main challenge lies in finding the correct offset for translation.

The concept for translation is already discussed in the other answers given here so I will explain how to get the right offset. Idea is that matching features in two images should have the same coordinate in the final stitched image.

Let's say we refer images as follows:

• 'source image' (si): the image which needs to be warped
• 'destination image' (di): the image to whose perspective 'source image' will be warped
• 'warped source image'(wsi): source image after warping it to the destination image perspective

Following is what you need to do in order to calculate offset for translation:

1. After you have sampled the good matches and found the mask from homography, store the best match's keypoint(one with a minimum distance and being an inlier (should get the value of 1 in mask obtained from homography calculation)) in si and di. Let's say best match's keypoint in si and diisbm_siandbm_di` respectively.

   bm_si = [x1, y1,1]

   bm_di = [x2, y2, 1]

2. Find the position of bm_si in wsi by simply multiplying it with the homography matrix (H). bm_wsi = np.dot(H,bm_si)

   bm_wsi = [x/bm_wsi[2] for x in bm_wsi]

3. Depending on where you will be placing the di on the output of si warping (=wsi), adjust the bm_di

   Let's say if you are warping from the left image to right image (such that left image is si and the right image is di) then you will placing di on the right side wsi and hence bm_di[0] += si.shape[0]

4. Now after the above steps

   x_offset = bm_di[0] - bm_si[0]

   y_offset = bm_di[1] - bm_si[1]

5. Using calculated offset find the new homography matrix and warp the si.

   T = np.array([[1, 0, x_offset], [0, 1, y_offset], [0, 0, 1]])

   translated_H = np.dot(T.H)

   wsi_frame_size = tuple(2*x for x in si.shape)

   stitched = cv2.warpPerspective(si, translated_H, wsi_frame_size)

   stitched[0:si.shape[0],si.shape[1]:] = di

---

this is my solution

since third parameter in "warpPerspective()" is a transformation matrix,

we can make a transformation matrix , which moves the image backward first ,then rotates the image,finally moves the image forward .

In my case,I have a image with height of 160 px and width of 160 px. I want to rotate the image around [80,80] instead of around [0,0]

first,moves the image backward (that means T1)

then rotates the image (that means R)

finally moves the image forward (that means T2)

void rotateImage(Mat &src_img,int degree)
{
float radian=(degree/180.0)*M_PI;
Mat R(3,3,CV_32FC1,Scalar(0));
R.at<float>(0,0)=cos(radian);R.at<float>(0,1)=-sin(radian);
R.at<float>(1,0)=sin(radian);R.at<float>(1,1)=cos(radian);
R.at<float>(2,2)=1;
Mat T1(3,3,CV_32FC1,Scalar(0));
T1.at<float>(0,2)=-80;
T1.at<float>(1,2)=-80;
T1.at<float>(0,0)=1;
T1.at<float>(1,1)=1;
T1.at<float>(2,2)=1;
Mat T2(3,3,CV_32FC1,Scalar(0));
T2.at<float>(0,2)=80;
T2.at<float>(1,2)=80;
T2.at<float>(0,0)=1;
T2.at<float>(1,1)=1;
T2.at<float>(2,2)=1;

std::cerr<<T1<<std::endl;
std::cerr<<R<<std::endl;
std::cerr<<T2<<std::endl;
std::cerr<<T2*R*T1<<"\n"<<std::endl;

cv::warpPerspective(src_img, src_img, T2*R*T1, src_img.size(), cv::INTER_LINEAR);
}

---

Here is a opencv-python solution for your problem, I put it on github: https://github.com/Sanster/notes/blob/master/opencv/warpPerspective.md

The key point is as user3094631 said, get two translation matrix(T1, T2) and apply to the Rotate matrix(M) T2*M*T1

In the code I give, T1 is from the center point of origin image, and T2 is from the left-top point of the transformed boundingBox. The transformed boundingBox comes from origin corner points:

height = img.shape[0]
width = img.shape[1]
#..get T1
#..get M
pnts = np.asarray([
    [0, 0],
    [width, 0],
    [width, height],
    [0, height]
    ], dtype=np.float32)
pnts = np.array([pnts])
dst_pnts = cv2.perspectiveTransform(pnts, M * T1)[0]
dst_pnts = np.asarray(dst_pnts, dtype=np.float32)
bbox = cv2.boundingRect(dst_pnts)
T2 = np.matrix([[1., 0., 0 - bbox[0]],
                [0., 1., 0 - bbox[1]],
                [0., 0., 1.]])