how to get PointNet (no pointnetplus) layers?

Kshitij Singh answered . 2025-03-28 12:16:40

To use the PointNet network in a format that can be utilized in the "DeepLearning HDL Toolbox" or "Deep Network Designer" in MATLAB, you need to work with the underlying layers of PointNet. Unfortunately, the PointNet model provided in the MATLAB documentation is typically defined as a Layered Network, not directly as a SeriesNetwork or DAGNetwork.

Here’s a step-by-step guide on how to convert the PointNet model into a SeriesNetwork or DAGNetwork that can be used in tools like DeepNetworkDesigner or DeepLearning HDL Toolbox.

Step 1: Load Pretrained PointNet Model

First, if you have already trained the PointNet model, you can load it using:

pointNetModel = load('pointnet_model.mat');  % Load trained model
net = pointNetModel.net;  % Extract the model

If you don’t have a pretrained model, you can train PointNet for classification or segmentation using the instructions from the PointNet for classification example.

Step 2: Convert PointNet to SeriesNetwork or DAGNetwork

MATLAB supports the use of SeriesNetwork and DAGNetwork for deployment and compatibility with tools like DeepLearning HDL Toolbox and Deep Network Designer. To convert the PointNet model, you need to manually create a SeriesNetwork or DAGNetwork using the layers that define PointNet’s architecture.

Convert to SeriesNetwork

A SeriesNetwork is a simpler type of network where each layer is connected sequentially (i.e., from one layer to the next without branching). You can convert your network layers into this format using:

layers = [
    % Input Layer (Example: Point Cloud Input Layer)
    imageInputLayer([num_points, 3, 1], 'Name', 'input', 'Normalization', 'none')

    % Transform layers (PointNet typically has transformations, i.e., T-Net)
    % Example: Fully Connected layers for transformations
    fullyConnectedLayer(64, 'Name', 'fc1')
    batchNormalizationLayer('Name', 'bn1')
    reluLayer('Name', 'relu1')

    % Further layers such as MLP (Multi-Layer Perceptron)
    fullyConnectedLayer(128, 'Name', 'fc2')
    batchNormalizationLayer('Name', 'bn2')
    reluLayer('Name', 'relu2')

    % Output Layer (Classifications or Segmentation outputs)
    fullyConnectedLayer(num_classes, 'Name', 'fc_out')
    softmaxLayer('Name', 'softmax')
    classificationLayer('Name', 'classoutput')
];

% Create SeriesNetwork from the layers
pointNetSeriesNet = assembleNetwork(layers);

Convert to DAGNetwork (for more complex architectures)

A DAGNetwork allows for more complex architectures, such as models with skip connections. For PointNet, you might need to incorporate branches or skips between layers (e.g., skip connections for multi-scale features in segmentation tasks).

layers = [
    % Input Layer
    imageInputLayer([num_points, 3, 1], 'Name', 'input', 'Normalization', 'none')

    % Transformations and shared multi-layer perceptron
    fullyConnectedLayer(64, 'Name', 'fc1')
    batchNormalizationLayer('Name', 'bn1')
    reluLayer('Name', 'relu1')

    % Skip connections or multiple branches if needed
    fullyConnectedLayer(128, 'Name', 'fc2')
    batchNormalizationLayer('Name', 'bn2')
    reluLayer('Name', 'relu2')

    % Output Layer
    fullyConnectedLayer(num_classes, 'Name', 'fc_out')
    softmaxLayer('Name', 'softmax')
    classificationLayer('Name', 'classoutput')
];

% Create DAGNetwork from the layers
pointNetDAGNet = DAGNetwork(layers);

Step 3: Export Model for HDL Toolbox

Once you have the network in the SeriesNetwork or DAGNetwork format, you can export it for use in the DeepLearning HDL Toolbox or Xilinx FPGA and SoC Device using the following:

% For SeriesNetwork
exportToHDL(pointNetSeriesNet, 'NetworkName', 'pointNetHDLModel');

% Or for DAGNetwork
exportToHDL(pointNetDAGNet, 'NetworkName', 'pointNetHDLModel');

Step 4: Open in Deep Network Designer

Once the model is converted, you can open it in Deep Network Designer:

% Open Deep Network Designer to inspect/edit the model
deepNetworkDesigner(pointNetSeriesNet);

This should help you visualize the network structure and make any further modifications needed.

Conclusion

To summarize:

1. Extract PointNet Layers: You can extract the layers from the original PointNet model.
2. Create a SeriesNetwork or DAGNetwork: Manually build a SeriesNetwork or DAGNetwork using the layers extracted from PointNet.
3. Export the Network: Export the network to be compatible with tools like DeepLearning HDL Toolbox or Deep Network Designer.
4. Use in FPGA/SoC: After conversion, you can deploy the network on FPGA/SoC using the DeepLearning HDL Toolbox.

By following these steps, you will be able to use the original PointNet model for FPGA deployment and other tools, making it compatible with Xilinx FPGA or SoC devices.

Not satisfied with the answer ?? ASK NOW