OiO.lk Community platform!

Oio.lk is an excellent forum for developers, providing a wide range of resources, discussions, and support for those in the developer community. Join oio.lk today to connect with like-minded professionals, share insights, and stay updated on the latest trends and technologies in the development field.
  You need to log in or register to access the solved answers to this problem.
  • You have reached the maximum number of guest views allowed
  • Please register below to remove this limitation

Decoding ROS messages from ROSBridge

  • Thread starter Thread starter Paul Brink
  • Start date Start date
P

Paul Brink

Guest
How can I decode rosbridge data in the browser?

So far I've been able to decode the following types:


  • Uncompressed raw RGB
  • Uncompressed raw Depth
  • JPEG compressed RGB

My Problem now is decoding compressed depth and PointCloud2 data. As far as my understanding goes, the data is encoded as base64. The depth image has been compressed to a mono16 PNG. I have tried many different approaches, but none seem to work. The depth image is supposed to contain 307200 depth values of 16 bits each.

I do not want to display this data in something like ros3djs or webviz (cloud not figure out how they do the decoding). I want to decode the data and use it in my own analysis.

Steps to reproduce:

Here is a sample file. It contains the data field of the JSON message: https://drive.google.com/file/d/18ZPpWrH9TKtPBbevfGdceZVpmmkiP4bh/view?usp=sharing

OR

  1. Make sure you have a device publising or a rosbag playing
  2. roslaunch rosbridge_server rosbridge_websocket.launch
  3. Launch your web page and make sure you added roslibjs in your script tag

The JS of my web page is simplified to this:

Code:
var ros = new ROSLIB.Ros({
    url: 'ws://127.0.0.1:9090'
});


var depthListener = new ROSLIB.Topic({
    ros: ros,
    name: '/camera/color/image_raw/compressedDepth',
    messageType: 'sensor_msgs/CompressedImage'
});

var pointCloudListener = new ROSLIB.Topic({
    ros: ros,
    name: '/camera/depth/color/points',
    messageType: 'sensor_msgs/PointCloud2'
});



depthListener.subscribe(function (message) {
    console.log(message);
    depthListener.unsubscribe();
});

pointCloudListener.subscribe(function (message) {
    console.log(message);
    pointCloudListener.unsubscribe();
});

I have set the two topics to unsubscribe after the first message so that my console does net get flooded.

Provided a screenshot of the console logs for depth image

Screenshot of depth image output

and for pointcloud

screenshot of pointcloud console log

This is what I have so far, but the onload function is never triggered.

Code:
image = new Image();
    image.src = "data:image/png;base64, " + message.data

    image.onload = function(){
        image.decode().then(() =>{
            if(image.width != 0 && image.height != 0){
                canvas.width = image.width;
                canvas.height = image.height;
                ctx = canvas.getContext('2d');
                ctx.drawImage(image, 0, 0);
                image_data = canvas.getContext('2d').getImageData(0,0, 640,480).data;
            }
        });
    }

I think this OpenCV code was used to compress the image. The message can essentially be thought of as a 16 bit gray scale image.

Please comment if I can update the question with specific information

<p><strong>How can I decode rosbridge data in the browser?</strong></p>
<p><strong>So far I've been able to decode the following types:</strong></p>
<ul>
<li>Uncompressed raw RGB</li>
<li>Uncompressed raw Depth</li>
<li>JPEG compressed RGB</li>
</ul>
<p>My Problem now is decoding compressed depth and PointCloud2 data. As far as my understanding goes, the data is encoded as base64. The depth image has been compressed to a mono16 PNG. I have tried many different approaches, but none seem to work. The depth image is supposed to contain 307200 depth values of 16 bits each.</p>
<p>I do not want to display this data in something like ros3djs or webviz (cloud not figure out how they do the decoding). I want to decode the data and use it in my own analysis.</p>
<p><strong>Steps to reproduce:</strong></p>
<p>Here is a sample file. It contains the data field of the JSON message: <a href="https://drive.google.com/file/d/18ZPpWrH9TKtPBbevfGdceZVpmmkiP4bh/view?usp=sharing" rel="nofollow noreferrer">https://drive.google.com/file/d/18ZPpWrH9TKtPBbevfGdceZVpmmkiP4bh/view?usp=sharing</a></p>
<p>OR</p>
<ol>
<li>Make sure you have a device publising or a rosbag playing</li>
<li>roslaunch rosbridge_server rosbridge_websocket.launch</li>
<li>Launch your web page and make sure you added roslibjs in your script tag</li>
</ol>
<p><strong>The JS of my web page is simplified to this:</strong></p>
<pre><code>var ros = new ROSLIB.Ros({
url: 'ws://127.0.0.1:9090'
});


var depthListener = new ROSLIB.Topic({
ros: ros,
name: '/camera/color/image_raw/compressedDepth',
messageType: 'sensor_msgs/CompressedImage'
});

var pointCloudListener = new ROSLIB.Topic({
ros: ros,
name: '/camera/depth/color/points',
messageType: 'sensor_msgs/PointCloud2'
});



depthListener.subscribe(function (message) {
console.log(message);
depthListener.unsubscribe();
});

pointCloudListener.subscribe(function (message) {
console.log(message);
pointCloudListener.unsubscribe();
});
</code></pre>
<p>I have set the two topics to unsubscribe after the first message so that my console does net get flooded.</p>
<p>Provided a screenshot of the console logs for depth image</p>
<p><a href="https://i.sstatic.net/vEXx5.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/vEXx5.jpg" alt="Screenshot of depth image output" /></a></p>
<p>and for pointcloud</p>
<p><a href="https://i.sstatic.net/dsANw.jpg" rel="nofollow noreferrer"><img src="https://i.sstatic.net/dsANw.jpg" alt="screenshot of pointcloud console log" /></a></p>
<p><strong>This is what I have so far, but the onload function is never triggered.</strong></p>
<pre><code>image = new Image();
image.src = "data:image/png;base64, " + message.data

image.onload = function(){
image.decode().then(() =>{
if(image.width != 0 && image.height != 0){
canvas.width = image.width;
canvas.height = image.height;
ctx = canvas.getContext('2d');
ctx.drawImage(image, 0, 0);
image_data = canvas.getContext('2d').getImageData(0,0, 640,480).data;
}
});
}
</code></pre>
<p>I think <a href="https://github.com/ros-perception/i...ssed_depth_image_transport/src/codec.cpp#L364" rel="nofollow noreferrer">this</a> OpenCV code was used to compress the image.
The message can essentially be thought of as a 16 bit gray scale image.</p>
<p>Please comment if I can update the question with specific information</p>
 

Latest posts

Top