Skip to content

The Final Project of team 8, NTUEE, Digital Circuits Lab (2021 Fall)

Notifications You must be signed in to change notification settings

DCLAB08/Real-Time-Image-Video-Dehazing

Repository files navigation

Real-Time-Image-Video-Dehazing

The aim of this project is to achieve real time dehazing for video (or image).

We utilize DE2-115 FPGA, ethernet connection, camera TRDB-D5M and display through VGA in this project.

With our own router, we achieve 10 FPS in RGB24 and 15 FPS in RGB16.

It;'s a short demo gif. The quality is not that good due to conversion from video to gif. demo

Block Diagram

image

The diagram shows the connection of ethernet mode. In camera mode, the data and valid line should be from RAW2RGB module.

Introduction

This project is the final project for NTUEE Digital Circuits Lab, 2021 Fall.

Contributor:

Features

  • Dehaze picture or video input via ethernet connection or camera (The algorithm is hardware-friendly)

You can visit CModel for more information

  • Achieve real time usage

Limitation

  • Currently only support 640x480 image input
  • For ethernet input, only support UDP protocol
  • For ethernet input, only support RGB24/RGB16 bit input

Usage

  • You can use either camera or ethernet as your image input (refer to step below)
  • Default ethernet
  • Connect VGA to display
  • Connect TRDB-D5M or ethernet wire to input image
  • Use switch 1 for displaying transmission map
  • Use switch 2 for displaying dehazed image
  • Button 0 to reset VGA & SDRAM & Camera
  • Button 1 to reset Ethernet and LCD

Ethernet Input 🔌

Run FPGA and send image (or video) by python

For ethernet & FPGA usage, we refer to alexforencich's project. Reference here

Make sure your PC (python) is in the same local network as your FPGA

FPGA

  • Please config src/Ethernet.v Line 200 and use your own IP address
  • Config RGB16 in config.h (Default RGB24)
  • ⚠️ Remember to config HOST and PORT in src/Ethernet.v file for your FPGA IP address

Python

  • python/send_video.py: send mp4 file (slower)
  • python/send_stream.py: send streaming image (HLS protocol) from provided URL (slower)

    stream only support RGB24

  • python/send_rawfile.py: send raw file (generated by video2file) (faster)
  • python/video2file.py: convert mp4 into rawfile
  • python/send_image.py: send jpg file
  • ⚠️ Remember to config HOST and PORT in each python file for your FPGA IP address

Because we use UDP protocol, please make sure your internet condition is good, otherwise some data might lost and cause display failure. In this case, you may increase the DELAY in python file to ensure the data was sent successfully (but lower FPS).


Camera Input 📷

Config DE2-115/config.h and define Camera

For TRDB-D5M & FPGA usage, we refer to Ahmed Abdelfattah's project. Reference here

Compile

  • For camera input, follow steps in "Camera Input" and compile through Quartus.
  • For ethernet input, make sure you modify IP address for FPGA and compile through Quartus.

📝 File needed for compilation

We use quartus to compile and generate the output files. Remember to import assignment (fpga.qsf) and add the following files to the path of your project.

  • all files in src/

  • all files in DE2-115/

  • verilog-ethernet/rtl/arp.v

  • verilog-ethernet/rtl/arp_cache.v

  • verilog-ethernet/rtl/arp_eth_rx.v

  • verilog-ethernet/rtl/arp_eth_tx.v

  • verilog-ethernet/rtl/axis_gmii_rx.v

  • verilog-ethernet/rtl/axis_gmii_tx.v

  • verilog-ethernet/rtl/eth_mac_1g_rgmii_fifo.v

  • verilog-ethernet/rtl/eth_mac_1g_rgmii.v

  • verilog-ethernet/rtl/eth_mac_1g.v

  • verilog-ethernet/rtl/eth_axis_rx.v

  • verilog-ethernet/rtl/eth_axis_tx.v

  • verilog-ethernet/rtl/eth_arb_mux.v

  • verilog-ethernet/rtl/eth_mux.v

  • verilog-ethernet/rtl/iddr.v

  • verilog-ethernet/rtl/ip.v

  • verilog-ethernet/rtl/ip_arb_mux.v

  • verilog-ethernet/rtl/ip_complete.v

  • verilog-ethernet/rtl/ip_eth_rx.v

  • verilog-ethernet/rtl/ip_eth_tx.v

  • verilog-ethernet/rtl/ip_mux.v

  • verilog-ethernet/rtl/lfsr.v

  • verilog-ethernet/rtl/oddr.v

  • verilog-ethernet/rtl/rgmii_phy_if.v

  • verilog-ethernet/rtl/ssio_ddr_in.v

  • verilog-ethernet/rtl/ssio_ddr_out.v

  • verilog-ethernet/rtl/udp_complete.v

  • verilog-ethernet/rtl/udp_checksum_gen.v

  • verilog-ethernet/rtl/udp.v

  • verilog-ethernet/rtl/udp_ip_rx.v

  • verilog-ethernet/rtl/udp_ip_tx.v

  • verilog-ethernet/lib/axis/rtl/arbiter.v

  • verilog-ethernet/lib/axis/rtl/axis_fifo.v

  • verilog-ethernet/lib/axis/rtl/axis_async_fifo.v

  • verilog-ethernet/lib/axis/rtl/axis_async_fifo_adapter.v

  • verilog-ethernet/lib/axis/rtl/priority_encoder.v

  • verilog-ethernet/lib/axis/rtl/sync_reset.v

  • fpga.sdc

Future work

  • Support jpg input format and implement decoder on FPGA
  • Support higher resolution and higher FPS
  • Solve display failure issue (Maybe use TCP instead or add check module on FPGA)

About

The Final Project of team 8, NTUEE, Digital Circuits Lab (2021 Fall)

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published