# [LiDAR_Inertial_Visual_Handhold](https://zhuanlan.zhihu.com/p/670136001)
### News
* **`15 July 2024`:** Fix the bug related to LiDAR (IMU) timestamp compensation in the `livox_ros_driver`, remove the dynamic library dependency from `mvs_ros_driver`, and enable this synchronization scheme to use the official `livox_sdk`.
* **`28 June 2024`:** Update reproduction videos and some important notes.
* **`27 May 2024`:** Fix a bug in the calculation of the synthetic GPRMC timestamp, which could cause timestamp rollback. Additionally, we add checksum verification for GPRMC.
* **`20 May 2024`:** According to Livox Avia's pin requirements, we convert PPS TTL level to RS485 level and supplement it with more detailed electronic connection and material list.
## 1. Introduction
This repository provides the **CAD files** (with suffix â\*.SLDPRT and \*.SLDASMâ) for our handheld device, which can be opened and edited with [*Solidworks*](https://www.solidworks.com). All of the modules are suitable for printing with [*FDM (Fused Deposition Modeling)*](https://en.wikipedia.org/wiki/Fused_filament_fabrication). In addition, we have also open-sourced our **hardware synchronization scheme**, as well as the **STM32 source code** and **hardware wiring configuration** instructions.
## 2. Relevant guidance video
Thanks to the Bilibili uploader [GundaSmart](https://space.bilibili.com/687639149?spm_id_from=333.788.0.0) for making a detailed reproduction video. Everyone can follow the video to reproduce it. His sensor configuration includes a MID360 and a Hikvision pinhole camera.
[FAST-LIVO Reproduction (Part 1)](https://www.bilibili.com/video/BV1T142197ci/?share_source=copy_web&vd_source=f77407286e8f0ee71b845d76498ada9d)
[FAST-LIVO Reproduction (Part 2)](https://www.bilibili.com/video/BV14T421X7m9/?share_source=copy_web&vd_source=f77407286e8f0ee71b845d76498ada9d)
[FAST-LIVO Reproduction (Part 3)](https://www.bilibili.com/video/BV16E42137ru/?share_source=copy_web&vd_source=f77407286e8f0ee71b845d76498ada9d)
[FAST-LIVO Reproduction (Part 4)](https://www.bilibili.com/video/BV1Yb421i7xK/?share_source=copy_web&vd_source=f77407286e8f0ee71b845d76498ada9d)
[FAST-LIVO Reproduction (Part 5)](https://www.bilibili.com/video/BV1Jz421b7px/?share_source=copy_web&vd_source=f77407286e8f0ee71b845d76498ada9d)
## 3. Guide to installation
### 3.1 Root directory
âââ handhold_cad/ - CAD source files
â âââ ...
âââ livox_ros_driver/ - Livox LiDAR ROS driver
â âââ ...
âââ mvs_ros_pkg/ - Camera driver
â âââ ...
âââ stm32_timersync-open/ - Embedded engineering folder
â âââ USER/ - Main functionality folder
â âââ ...
âââ README.md - Project homepage document
âââ ...
### 3.2 Assembly instruction
The assembly instructions are demonstrated in the GIFs below. Each module is clearly labeled corresponding to the name of its STL file.
### 3.3 Electronic connection
The guide for the electronic connections is presented as follows:
Livox AVIA LiDAR M12 |
Peripheral Function |
Diagram |
PIN 1 |
Power: + |
AVIA 12-Pin Interface, top: female bottom: male
|
PIN 2 |
Ground |
PIN 7 |
Ethernet: RX- |
PIN 6 |
Ethernet: RX+ |
PIN 5 |
Ethernet: TX- |
PIN 4 |
Ethernet: TX+ |
PIN 11 (Sync-) |
RS485_Output B- |
PIN 12 (Sync+) |
RS485_Output A+ |
MVS Camera 6PIN |
Name |
I/O Type |
Description |
Peripheral Function |
Diagram |
PIN 1 |
DC_PWR |
-- |
Power Supply |
|
MVS Camera 6-Pin Interface
|
PIN 2 |
OPTO_IN |
Line 0+ |
Optical Isolation Input |
STM32 PA1 |
PIN 3 |
GPIO |
Line 2+ |
General Purpose Input/Output |
|
PIN 4 |
OPTO_OUT |
Line 1+ |
Optical Isolation Output |
|
PIN 5 |
OPTO_GND |
Line 0- / 1- |
Optical Isolation Ground |
STM32 GND |
PIN 6 |
GND |
Line 2- |
Ground |
|
STM32 |
Peripheral Function |
PA1 |
MVS camera PIN2 (OPTO_IN) |
PB5 |
RS485_Input (TXD) |
TXD |
TTL2USB (RXD) |
VCC |
RS485_Input (VCC) | TTL2USB (VCC) |
GND |
RS485_Input (GND) | TTL2USB (GND) | MVS camera PIN5 (GND) |
RS485 |
Peripheral Function |
RS485_Output A+ |
LiDAR M12 PIN12 (Sync+) |
RS485_Output B- |
LiDAR M12 PIN11 (Sync-) |
RS485_Output (GND) |
LiDAR M12 PIN2 (GND) |
**ð´ð¥ IMPORTANT:**
* **STM32 PB5 (PPS signal) is converted from TTL to RS-485, resulting in RS485_Output A+ and RS485_Output B-;**
* **STM32 TXD (GPRMC) is converted from TTL to USB and sent to the PC;**
* **If you are using the Mid360, you can directly connect STM32 PB5 to LiDAR M12 pps interface;**
* **Before you `roslaunch livox_ros_driver livox_lidar_msg.launch`, you can use `sudo chmod a+rw /dev/ttyUSB0` grant permissions to the USB serial port;**
* **The default connection method in this repository directly connects to M12 without using the Livox converter. If you use the converter, you can connect STM32 PB5 (PPS signal) to the Livox converter Sync Port without converting TTL to 485 level. Refer to [Issue 19](https://github.com/sheng00125/LIV_handhold/issues/19) for details.**
## 4. How to run the driver
```bash
roslaunch mvs_ros_pkg mvs_camera_trigger.launch
roslaunch livox_ros_driver livox_lidar_msg.launch
rosbag record /livox/lidar /livox/imu left_camera/image
```
## 5. Main material lists (only for reference)
| Item | Pics | Purchasing list |
| :------------: | :------------: | :------------: |
| Livox Avia LiDAR |
| [Livox Avia](https://store.dji.com/hk-en/product/livox-avia) |
| CMOS |
| [MV-CA013-21UC ](https://www.hikrobotics.com/en/machinevision/productdetail?id=1314&pageNumber=1&pageSize=50) |
| Camera Len |
| [ MVL-HF0628M-6MPE](https://m.tb.cn/h.gXmtLRX2UYzGDzH?tk=hIS7WGPOY0y) |
| STM32 |
| [STM32F103C8T6](https://m.tb.cn/h.ggkS9Kp?tk=orRfWz6M784) |
| Screen |
| [IPS Screen 10''](https://m.tb.cn/h.ggkhQ7e?tk=LntBWz6mHDL) |
| Battery |
| [4800mah](https://m.tb.cn/h.g5vJI7a?tk=ofKdWz6OYQm) |
| TTL to USB | <