NVIDIA Generative AI Multimodal Sample Questions:

1. You are building a multimodal emotion recognition system that combines facial expressions (images) and spoken language (audio). The image data is preprocessed using a CNN, and the audio data is processed using an LSTM. Which of the following fusion strategies would be MOST effective for combining these two modalities to predict the emotion?

A) Late fusion by training separate classifiers on the CNN and LSTM outputs and then averaging their predicted probabilities.
B) Intermediate fusion by concatenating the CNN and LSTM hidden state representations before feeding them into a shared classification layer.
C) Training the CNN and LSTM models independently without any fusion.
D) Using an attention mechanism to weigh the contributions of the CNN and LSTM features based on their relevance to the predicted emotion.
E) Early fusion by concatenating the raw pixel values of the images with the raw audio waveform.

2. You are analyzing the performance of a Generative A1 model and notice that it is overfitting to the training dat a. Which techniques can you apply to mitigate overfitting and improve the model's generalization performance? Select all that apply:

A) Use dropout layers during training.
B) Add L1 or L2 regularization to the model's loss function.
C) Increase the learning rate.
D) Increase the size of the training dataset.
E) Decrease the model's complexity (e.g., reduce the number of layers or parameters).

3. You're building a multimodal model that takes images and text as input. You notice that your model is heavily biased towards the text modality, essentially ignoring the visual input. Which of the following strategies could you employ to address this modality imbalance? (Select TWO)

A) Implement a gating mechanism that dynamically adjusts the contribution of each modality based on the input.
B) Reduce the size of the visual encoder.
C) Remove the text modality entirely.
D) Increase the learning rate for the text encoder.
E) Use a modality-specific loss function, weighting the loss from the visual modality more heavily.

4. You are fine-tuning a pre-trained large language model (LLM) for a specific text generation task using LoRA (Low-Rank Adaptation).
Which of the following statements accurately describes the benefits and limitations of using LoRA?

A) LoRA can improve the accuracy of the fine-tuned model compared to full fine-tuning by preventing overfitting.
B) LoRA allows for efficient task switching by only storing and loading the small LoRA parameters for different tasks, while keeping the original LLM weights frozen.
C) A and B.
D) LoRA reduces the number of trainable parameters by inserting low-rank matrices into the original model layers, making fine-tuning more memory-efficient.
E) LoRA is not compatible with model parallelism techniques.

5. You are tasked with creating a multimodal AI application that analyzes social media posts containing text, images, and user profile information to predict the likelihood of a post going viral. Which feature engineering techniques are most effective for representing and integrating these different modalities?

A) Using a combination of TF-IDF for text, pixel values for images, and numerical features for user profile information. Then apply PCA for dimensionality reduction.
B) Using TF-IDF for text, pixel values for images, and one-hot encoding for user profile information.
C) Using character-level n-grams for text, edge detection for images, and boole an features for user profile information.
D) Using word embeddings (e.g., Word2Vec, GloVe) for text, pre-trained CNN features (e.g., from ResNet, Inception) for images, and embedding user profiles using a graph embedding technique.
E) Using bag-of-words for text, histogram of oriented gradients (HOG) for images, and simple numerical features (e.g., number of followers) for user profiles.

Solutions: