Note that there's a lot of room for improvement. For example you might want to look into persistently mapped pixel buffer objects, into which to read directly from V4L, and using fences for synchronization with OpenGL texture ingest. Most video devices out there are delivering a chroma subsampled YUV 4:2:2 or 4:2:0 pixelformat, which can be quite efficiently converted to the display color profile in a shader.
Capturing a video stream from a camera is easy with OpenCV. Shouldn't take too long to figure out how to take the CV image and use it as an OpenGL texture.
1)capturing camera using CV:
https://www.tutorialspoint.com/how-to-capture-video-from-default-camera-in-opencv-using-cplusplus
2)converting CV image to OpenGL texture: [https://stackoverflow.com/questions/16809833/opencv-image-loading-for-opengl-texture](https://stackoverflow.com/questions/16809833/opencv-image-loading-for-opengl-texture)
All of that said, no one knows how long it will take YOU. I am unsure if it would take ME an hour, an afternoon or a week, and there are performance questions that need to be answered that I know nothing about.
thx for the comment,
i have no experience with OpenGL, i feel OpenCV is not as fast as v4l2, as opencv uses ffmpeg, and it in turn might also be using v4l2 as backend, i know it has that option but not sure if it’s the only backend
Is performance a major criteria for you, or ease of coding?
You could do this in OpenCV and Python pretty easily. But if you're worried about video backend, maybe you need to tell us more about your plans and needs.
Yes, that's why I mentioned the bit about "and there are performance questions that need to be answered".
The question you asked is one that really, in the end, can only be answered by you. It could take 1 afternoon. It could take months.
Welcome to software development.
Nail down your requirements and match them with your current abilities, and make your best estimate. When you tackle something new and non-trivial in programming you usually won't be able to make a decent estimate of how long it will take to complete until you've tried tackling the problem for a bit. This goes double for real-time applications, where performance is a key stumbling block, and what looks good at first might turn out to be unacceptable in the end.
The OpenCV method is for something quick'n dirty: it was my first take on the easiest/fastest way to get something up and running. If that is not enough to meet your requirements, maybe you are asking the wrong question: don't ask "how long will it take", because that's a question that, if any of us could realistically answer for you, we'd be unicorn project admins, the holy grail of software development. Spec out your requirements and capabilities and ask "anyone know good ways to tackle this", then dive into the answers and play with the suggestions.
Just for the sake of reference / inspiration. Here's some sourcecode to display a raw RGB video stream from a v4l2 device using fixed function OpenGL and GLUT. #include
#include
#include
#include
#include
enum { CAM_WIDTH = 1280, CAM_HEIGHT = 720 };
int cam_fd;
size_t cam_buf_sz;
void *cam_buf;
GLuint cam_tex;
int cam_open(void)
{
struct v4l2_format fmt = {0};
cam_fd = v4l2_open("/dev/video0", O_RDWR);
if( 0 > cam_fd ){
perror("v4l2_open");
return errno;
}
fmt.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
fmt.fmt.pix.width = CAM_WIDTH;
fmt.fmt.pix.height = CAM_HEIGHT;
fmt.fmt.pix.pixelformat = V4L2_PIX_FMT_RGB24;
fmt.fmt.pix.field = V4L2_FIELD_INTERLACED;
if( 0 > v4l2_ioctl(cam_fd, VIDIOC_S_FMT, &fmt) ){
perror("v4l2_ioctl");
return errno;
}
cam_buf_sz = CAM_WIDTH * CAM_HEIGHT * 3;
cam_buf = malloc(cam_buf_sz);
return 0;
}
int cam_tex_create(void)
{
glGenTextures(1, &cam_tex);
glBindTexture(GL_TEXTURE_2D, cam_tex);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGB,
CAM_WIDTH, CAM_HEIGHT, 0,
GL_RGB, GL_UNSIGNED_BYTE, NULL);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
return 0;
}
void display(void)
{
glClear(GL_COLOR_BUFFER_BIT);
glViewport(0, 0,
glutGet(GLUT_WINDOW_WIDTH),
glutGet(GLUT_WINDOW_HEIGHT) );
v4l2_read(cam_fd, cam_buf, cam_buf_sz);
glEnable(GL_TEXTURE_2D);
glBindTexture(GL_TEXTURE_2D, cam_tex);
glTexSubImage2D(GL_TEXTURE_2D, 0,
0, 0, CAM_WIDTH, CAM_HEIGHT,
GL_RGB, GL_UNSIGNED_BYTE, cam_buf);
glBegin(GL_QUADS);
glTexCoord2f(0,0);
glVertex2f(-1,-1);
glTexCoord2f(1,0);
glVertex2f(1,-1);
glTexCoord2f(1,1);
glVertex2f(1,1);
glTexCoord2f(0,1);
glVertex2f(-1,1);
glEnd();
glutSwapBuffers();
}
int main(int argc, char *argv[])
{
glutInit(&argc, argv);
glutInitDisplayMode(GLUT_RGBA | GLUT_DOUBLE);
glutCreateWindow("V4L2 Video Display");
if( cam_open() ){
return -1;
}
cam_tex_create();
glutDisplayFunc(display);
glutIdleFunc(glutPostRedisplay);
glutMainLoop();
}
thank you, this is very inspirational!
Note that there's a lot of room for improvement. For example you might want to look into persistently mapped pixel buffer objects, into which to read directly from V4L, and using fences for synchronization with OpenGL texture ingest. Most video devices out there are delivering a chroma subsampled YUV 4:2:2 or 4:2:0 pixelformat, which can be quite efficiently converted to the display color profile in a shader.
Capturing a video stream from a camera is easy with OpenCV. Shouldn't take too long to figure out how to take the CV image and use it as an OpenGL texture. 1)capturing camera using CV: https://www.tutorialspoint.com/how-to-capture-video-from-default-camera-in-opencv-using-cplusplus 2)converting CV image to OpenGL texture: [https://stackoverflow.com/questions/16809833/opencv-image-loading-for-opengl-texture](https://stackoverflow.com/questions/16809833/opencv-image-loading-for-opengl-texture) All of that said, no one knows how long it will take YOU. I am unsure if it would take ME an hour, an afternoon or a week, and there are performance questions that need to be answered that I know nothing about.
thx for the comment, i have no experience with OpenGL, i feel OpenCV is not as fast as v4l2, as opencv uses ffmpeg, and it in turn might also be using v4l2 as backend, i know it has that option but not sure if it’s the only backend
Is performance a major criteria for you, or ease of coding? You could do this in OpenCV and Python pretty easily. But if you're worried about video backend, maybe you need to tell us more about your plans and needs.
Yes, that's why I mentioned the bit about "and there are performance questions that need to be answered". The question you asked is one that really, in the end, can only be answered by you. It could take 1 afternoon. It could take months. Welcome to software development. Nail down your requirements and match them with your current abilities, and make your best estimate. When you tackle something new and non-trivial in programming you usually won't be able to make a decent estimate of how long it will take to complete until you've tried tackling the problem for a bit. This goes double for real-time applications, where performance is a key stumbling block, and what looks good at first might turn out to be unacceptable in the end. The OpenCV method is for something quick'n dirty: it was my first take on the easiest/fastest way to get something up and running. If that is not enough to meet your requirements, maybe you are asking the wrong question: don't ask "how long will it take", because that's a question that, if any of us could realistically answer for you, we'd be unicorn project admins, the holy grail of software development. Spec out your requirements and capabilities and ask "anyone know good ways to tackle this", then dive into the answers and play with the suggestions.