# Last Installation log

Install Jetpack 4.4 - With the changed Pinmux configuration
Download: SRMobileRobot, Camera and Pytorch-Nested file.
Install QT Creator - sudo apt-get install qt5-default qtcreator -y
Install JetsonGPIO - JetsonGPIO(C++)
Install Protobuf - C++ 3.19 version

./configure --prefix=/usr
make
make check
sudo make install
sudo ldconfig # refresh shared library cache.

1
2
3
4
5

Install doxygen - https://www.doxygen.nl/manual/install.html

git clone https://github.com/doxygen/doxygen.git
cd doxygen
mkdir build
cd build
cmake -Dbuild_wizard=YES ..
make
sudo make install

1
2
3
4
5
6
7

OpenCV 4.5.3

Obs.: Change script to version 4.5.3

# Installation

OpenCV 4.5.3

Obs.: Change script to version 4.5.3

Protobuf 3.18

TIP

sudo apt-get install qt5-default qtcreator -y

QtCreator Doxygen

sudo apt-get install qtcreator-dev

i2c-tools

ONNX

Generate the build dependencies for ONNX - Generate the ONNX protobuf source files

BASEDIR=$(pwd)
cd $BASEDIR
export ONNX_ML=1 #To clone ONNX with its ML extension
git clone --recursive https://github.com/onnx/onnx.git
unset ONNX_ML
cd onnx
git checkout 553df22c67bee5f0fe6599cff60f1afc6748c635


export LD_LIBRARY_PATH=/usr/lib:$LD_LIBRARY_PATH

/usr/bin/protoc onnx/onnx.proto --proto_path=. --proto_path=/usr/include/google/protobuf --cpp_out $BASEDIR/onnx/onnx

1
2
3
4
5
6
7
8
9
10
11
12

On Cmake:

include_directories("/home/thiago/Downloads/onnx/onnx")

# Installation for the Python Training

pip install albumentations==0.4.4 pip install sklearn

# Robot Programming

NVIDIA Jetson AGX Xavier GPIO Header Pinout

I2C - AGX

I2C - AGX - Github

Jetson Nano I2C PWM Servo Motors

ServoKit - Github

Driver

A driver for the Adafruit 16-Channel 12-bit PWM/Servo Driver - I2C interface - PCA9685.

ESCON Regulation Tuning - Expert Tuning Mode

# Install OpenCV with Cuda Enabled

This is need to use #include <opencv4/opencv2/cudawarping.hpp> like this:

    cv::cuda::resize(gpu_frame, resized, input_size, 0, 0, cv::INTER_NEAREST);

Link to the solution

Jetson OpenCV WITH_CUDA=ON

# Structure of C++ CMake project

link

# Maxon ESCON 50/5

PWM input frequency <-> PWM output frequency

A PWM input signal in use as the set value can command the ESCON's current or speed control. The PWM input in use as a set value is typically in use if the ESCON is commanded by a microcontroller which might have no analog output signal but almost ever a PWM output.

Restriction 1: 10 Hz ... 5 kHz
A good compromise in practice offering fast reaction and precise duty cycle measurement is to use 1 kHz which needs 1 ms for signal processing.

# PWM

JetsonGPIO(C++)

# GPIO Programming

Configuring the GPIO for raspberry pie

Discussion - Hardware PWM update with Jetson.GPIO

Command	Function
cat /sys/kernel/debug/gpio	list gpio active pins
echo 344	sudo tee /sys/class/gpio/export
ls /sys/class/gpio	list the classes
echo out >/sys/class/gpio/gpio344/direction	sets direction for output or input
echo 1 >/sys/class/gpio/gpio344/value	sets pin to hi
cat /sys/kernel/debug/tegra_pinctrl_reg	pinctrl reg

# Configuring GPIO

Configuring the 40-Pin Expansion Header

# Install OPenCV 3.31

Article from Econsystems

Github from econsystems

# QKeyEvent

Youtube

# h5 to .pb to .onnx to .engine

NVIDIA Article

Article

how-to-run-inference-using-tensorrt-c-api

# Pins

Setup Expansion head

# Flashing the jetson

Flash with Jetpack 4.4

# QTCreator

Install QTcreator

sudo apt-get install qt5-default qtcreator -y

# OpenCV

Install OpenCV

# 40 PIN configuration

https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/hw_setup_jetson_io.html

sudo /opt/nvidia/jetson-io/jetson-io.py

# Pinmux

pinmux

from: article

Tensor RT supports the following layer types.

Convolution: 2D Activation: ReLU, tanh and sigmoid Pooling: max and average ElementWise: sum, product or max of two tensors LRN: cross-channel only Fully-connected: with or without bias SoftMax: cross-channel only Deconvolutio

# GPIO

Install GPIO

Installation:

git clone https://github.com/pjueon/JetsonGPIO
cd JetsonGPIO/build
make all
sudo make install

Setting User Permissions:

sudo groupadd -f -r gpio
sudo usermod -a -G gpio your_user_name


cd ..
cd ..

sudo cp JetsonGPIO/99-gpio.rules /etc/udev/rules.d/

sudo udevadm control --reload-rules && sudo udevadm trigger

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

Compile the p[rogram:

g++ -o your_program_name your_source_code.cpp -lJetsonGPIO

# .pb to onnx

on the folder above the saved model folder:

python -m tf2onnx.convert --saved-model ./saved_model --output model.onnx

Github

# Quantization

Quantization

# Onnx to TensorRT engine

Onnx to TensorRT engine

onnx-tensorrt Parser

tfliteTENORRT

# save to tensorflow saved model


model.load_weights(SAVED_MODEL)
model.save("./saved_model")

1
2
3
4

# Using saved_model with CPP

using saved_model with CPP

# Inference using Tflite

Installing tflite

Inference using Tflite

built tflite with cmake

Installing tflite with cmake

TensorFlow->TensorRT Image Classification

TensorFlow/TensorRT Models on Jetson

# I2C

I2c Library for jetson

i2C video for arduino

Arduino Nano code:

// Library for I2C Bus
#include <Wire.h>

// Define Slave I2C Address
#define SLAVE_ADDR 9

// Define Slave answer size
#define ANSWERSIZE 5

int analogPinVelocity = A0;
int analogPinCurrent = A1;

// Define string with response to Master
String answer = "";




void setup(){

  // Initialize I2C Communications as Master
  Wire.begin(SLAVE_ADDR);


  // Function to run when data requested from master
  Wire.onRequest(requestVelocityA);

  // Function to run when data received from master
//  Wire.onReceive(receiveEvent);

  // Setup Serial Monitor
//  Serial.begin(9600);
//  Serial.printl("I2C Slave demonstration");
  
}
//
//void receiveEvent(){
//  while (0 < Wire.available()){
//    byte x = Wire.read();
//  }
//
//  //Print to serial monitor
//  Serial.println("Receive event");
//}

void requestVelocityA(){
  answer = analogRead(analogPinVelocity);
  
  // Setup byte variable in the corrent size
  byte response[ANSWERSIZE];

  // Format answer as array
  for (byte i=0; i<ANSWERSIZE; i++){
    response[i] = (byte)answer.charAt(i);
  }

  //Send response back to Master
  Wire.write(response, sizeof(response));

  // Print to serial monitor
//  Serial.println(response);
}

void loop(){
  // Time delay in loop
  delay(50);
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67

# Detect I2C

Find I2C Jetson

# I2C

I2C - AGX

I2C - AGX - Github

I2C library = libi2c-dev Kernel Documentation

i2c-tools

sudo apt-get install i2c-tools
sudo apt-get install libi2c-dev
sudo i2cdetect -y 2    //to see the address
sudo i2cdetect -r 2    //I will probe file /dev/i2c-2 using receive byte commands. I will probe address range 0x03-0x77.

1
2
3
4

amaork/libi2c - Good explanation

I2C on Raspberry py and arduino link

i2cdetect -y 1
i2cget -y 2 0x08

1
2

I2C for controleverything.com link

How To Install "libi2c-dev" Package on Ubuntu link

The SMBus

I2C library = libi2c-dev - Kernel Documentation

SMBus - Kernel Documentation

# SMBus Read Word

Implemented by i2c_smbus_read_word_data()

This operation is very like Read Byte; again, data is read from a device, from a designated register that is specified through the Comm byte. But this time, the data is a complete word (16 bits):

S Addr Wr [A] Comm [A] S Addr Rd [A] [DataLow] A [DataHigh] NA P

1
2

Functionality flag: I2C_FUNC_SMBUS_READ_WORD_DATA

Note the convenience function i2c_smbus_read_word_swapped() is available for reads where the two data bytes are the other way around (not SMBus compliant, but very popular.)

A word contains 32 bits. There are 65,536 different unsigned 16-bit numbers. The smallest unsigned 16-bit number is 0 and the largest is 65535.

I2C Usage - link

# usmbus

sudo apt update
sudo apt install libi2c-dev -y

1
2

# I2C for Raspberry Pi

link

# New answers from E-con

Dear Thiago,

We were able to recreate the issue and it looks like updating the pin function using the GPIO library updates the DTB which makes the camera support disabled. So, if you wish to change the GPIO header pin functions it is better to update the default pinmux.config which defines those pin states using the sample steps provided in the following link, https://elinux.org/Jetson/AGX_Xavier_CAN#Update_Pinmux

After flashing the Jetson AGX Xavier kit with the required pinmux settings, you can flash the STURDeCAM20_CUXVR package to work with the camera.

Note: The steps mentioned in the link are just a sample. Kindly use it as a reference and update the default pinmux.config.

link

# Updating the PWM pin - hard motherfucker

This happens all in the host: We are going to change the .cfg file on the JetPack folder. and then install on the Jetson. For this we have to download a excel file, change the pins configuration, create 3 dtsi files, copy them to the folder that have the script and execute the script. this script will generate the cfg file from the dtsi files. then we substitute the cfg file in the JetPack 4.4 folder.

on jetson nano

after changing the pins 15 and 18 to PWM

and copying the files to /home/thiago/nvidia/nvidia_sdk/JetPack_4.4_Linux_JETSON_AGX_XAVIER/Linux_for_Tegra/kernel/pinmux, we execute the script below to generate the .cfg file

Change the file name to /home/thiago/nvidia/nvidia_sdk/JetPack_4.4_Linux_JETSON_AGX_XAVIER/Linux_for_Tegra/kernel/pinmux/t19x/tegra19x-mb1-pinmux-p2888-0000-a04-p2822-0000-b01.cfg as per this discriptionhttps://elinux.org/Jetson/AGX_Xavier_CAN#Locate_Pinmux_configure_file.

Then copy the file to here /Xavier/Linux_for_Tegra/bootloader/t186ref/BCT

python pinmux-dts2cfg.py addr_info.txt gpio_addr_info.txt por_val.txt --mandatory_pinmux_file mandatory_pinmux.txt tegra19x-jetson_agx_devkit-pinmux.dtsi tegra19x-jetson_agx_devkit-gpio-default.dtsi 1.0 > galen.cfg

# UNET++ Pytorch

Github repository

Download the repository
copy the dataset to the input folder
Run

python train.py --dataset weldset --arch NestedUNet --img_ext .jpg --mask_ext .png

if you get this error: Check spell

num_classes: 1
input_w: 96
input_h: 96
loss: BCEDiceLoss
dataset: welset
img_ext: .jpg
mask_ext: .png
optimizer: SGD
lr: 0.001
momentum: 0.9
weight_decay: 0.0001
nesterov: False
scheduler: CosineAnnealingLR
min_lr: 1e-05
factor: 0.1
patience: 2
milestones: 1,2
gamma: 0.6666666666666666
early_stopping: -1
num_workers: 4
--------------------
=> creating model NestedUNet
Traceback (most recent call last):
  File "train.py", line 354, in <module>
    main()
  File "train.py", line 250, in main
    train_img_ids, val_img_ids = train_test_split(img_ids, test_size=0.2, random_state=41)
  File "C:\Python38\lib\site-packages\sklearn\model_selection\_split.py", line 2175, in train_test_split
    n_train, n_test = _validate_shuffle_split(n_samples, test_size, train_size,
  File "C:\Python38\lib\site-packages\sklearn\model_selection\_split.py", line 1857, in _validate_shuffle_split
    raise ValueError(
ValueError: With n_samples=0, test_size=0.2 and train_size=None, the resulting train set will be empty. Adjust any of the aforementioned parameters.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

Now create toonnx.py from val.py changing this:

import argparse
import os
from glob import glob

import cv2
import torch
import torch.backends.cudnn as cudnn
import yaml
from albumentations.augmentations import transforms
from albumentations.core.composition import Compose
from sklearn.model_selection import train_test_split
from tqdm import tqdm

import archs
from dataset import Dataset
from metrics import iou_score
from utils import AverageMeter


def parse_args():
    parser = argparse.ArgumentParser()

    parser.add_argument('--name', default=None,
                        help='model name')

    args = parser.parse_args()

    return args


def main():
    args = parse_args()

    with open('models/%s/config.yml' % args.name, 'r') as f:
        config = yaml.load(f, Loader=yaml.FullLoader)

    print('-'*20)
    for key in config.keys():
        print('%s: %s' % (key, str(config[key])))
    print('-'*20)

    cudnn.benchmark = True

    # create model
    print("=> creating model %s" % config['arch'])
    model = archs.__dict__[config['arch']](config['num_classes'],
                                           config['input_channels'],
                                           config['deep_supervision'])

    model = model.cuda()

    dummy_input = torch.randn(10, 3, 96, 96, device='cuda')

    torch.onnx.export(model, dummy_input, "Pytorch-NestedUnet.onnx", opset_version=11)



    # # Data loading code
    # img_ids = glob(os.path.join('inputs', config['dataset'], 'images', '*' + config['img_ext']))
    # img_ids = [os.path.splitext(os.path.basename(p))[0] for p in img_ids]

    # _, val_img_ids = train_test_split(img_ids, test_size=0.2, random_state=41)

    # model.load_state_dict(torch.load('models/%s/model.pth' %
    #                                  config['name']))
    # model.eval()

    # val_transform = Compose([
    #     transforms.Resize(config['input_h'], config['input_w']),
    #     transforms.Normalize(),
    # ])

    # val_dataset = Dataset(
    #     img_ids=val_img_ids,
    #     img_dir=os.path.join('inputs', config['dataset'], 'images'),
    #     mask_dir=os.path.join('inputs', config['dataset'], 'masks'),
    #     img_ext=config['img_ext'],
    #     mask_ext=config['mask_ext'],
    #     num_classes=config['num_classes'],
    #     transform=val_transform)
    # val_loader = torch.utils.data.DataLoader(
    #     val_dataset,
    #     batch_size=config['batch_size'],
    #     shuffle=False,
    #     num_workers=config['num_workers'],
    #     drop_last=False)

    # avg_meter = AverageMeter()

    # for c in range(config['num_classes']):
    #     os.makedirs(os.path.join('outputs', config['name'], str(c)), exist_ok=True)
    # with torch.no_grad():
    #     for input, target, meta in tqdm(val_loader, total=len(val_loader)):
    #         input = input.cuda()
    #         target = target.cuda()

    #         # compute output
    #         if config['deep_supervision']:
    #             output = model(input)[-1]
    #         else:
    #             output = model(input)

    #         iou = iou_score(output, target)
    #         avg_meter.update(iou, input.size(0))

    #         output = torch.sigmoid(output).cpu().numpy()

    #         for i in range(len(output)):
    #             for c in range(config['num_classes']):
    #                 cv2.imwrite(os.path.join('outputs', config['name'], str(c), meta['img_id'][i] + '.jpg'),
    #                             (output[i, c] * 255).astype('uint8'))

    # print('IoU: %.4f' % avg_meter.avg)

    torch.cuda.empty_cache()


if __name__ == '__main__':
    main()

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119

maybe you will get this error

RuntimeError: ONNX export failed: Couldn't export operator aten::upsample_bilinear2d

issue

Now go from onnx to TensorRT

ThiagoSoutoGit/Pytorch-Nested-UNet-to-Tensor-RT

Quantization

https://pytorch.org/docs/stable/torch.quantization.html?highlight=quantization#module-torch.quantization

# Aurtomatic mixed Precision FP32

INTRODUCTION TO MIXED PRECISION TRAINING

This repository holds NVIDIA-maintained utilities to streamline mixed precision and distributed training in Pytorch. Some of the code here will be included in upstream Pytorch eventually. The intention of Apex is to make up-to-date utilities available to users as quickly as possible.

Amp: Automatic Mixed Precision - Github