Iray Programmer's Manual

Implementation of an importer

This topic introduces:

  1. Basic concepts about the implementation and usage of custom importers used in conjunction with the Iray API:
  2. An example importer program consisting of vanilla_importer.h and example_importer.cpp. The importer is called Vanilla. For simplicity, it is defined in and used by the main application. The importer is intended as an illustrative skeleton: It implements all interfaces but does not parse the file content in a meaningful way.

Implementing an importer

The implementation of the Vanilla importer is structured in three parts:

Instances of mi::neuraylib::IImpexp_state are used to pass information about the current importer state, for example to recursive calls of importers. The Vanilla importer does not need to carry around any additional information besides what is required by the interface, therefore this simple implementation is fine. Note that the simple derivation from mi::base::Interface_implement suffices here (in contrast to example_plugins.cpp).

The Vanilla importer is given in the implementation of the mi::neuraylib::IImporter interface. Later, an instance of this class is registered as an importer with the Iray API. Most of the methods implemented here are defined in the base interface mi::neuraylib::IImpexp_state, as they are common for importers and exporters. The Vanilla importer claims to handle files with the extension .vnl and .van. It does not require specific capabilities of the reader to handle these formats. However, if the reader supports the lookahead capability, it will use a magic header check instead of relying on file name extensions.

The actual work of the Vanilla importer occurs in the import_elements() method. It is split into three parts:

  1. Creating an import result object
  2. Creating a group object which acts as the root group
  3. Reading the file line by line

While performing these tasks, the example program demonstrates what type of errors to detect, how errors can be reported, and how to implement an include file mechanism, and similar things, with a recursive call to the mi::neuraylib::IImport_api::import_elements() method.

Registering an importer

Registering importers is similar to registering user-defined classes. However, since importers are different from regular classes (for example, you cannot create instances of them using mi::neuraylib::ITransaction::create()); you need to use a registration method specific to importers. This registration method expects a pointer to an instance of the custom importer.

To run the example, you need to create two files called test1.vnl and test2.van, each five lines long and starting with a line saying VANILLA. For demonstration purposes, the example will print the generated error messages and the names of the imported elements.

vanilla_importer.h

001 /******************************************************************************
002  * © 1986, 2015 NVIDIA ARC GmbH. All rights reserved.
003  *****************************************************************************/
004 
005 #include <mi/neuraylib.h>
006 
007 #include <string>
008 #include <sstream>
009 
010 // Support function to handle mi::base::Message_severity to std::string conversion
011 static std::string enum_to_str( mi::base::Message_severity severity)
012 {
013     switch( severity) {
014         case mi::base::MESSAGE_SEVERITY_FATAL:
015             return "fatal";
016         case mi::base::MESSAGE_SEVERITY_ERROR:
017             return "error";
018         case mi::base::MESSAGE_SEVERITY_WARNING:
019             return "warning";
020         case mi::base::MESSAGE_SEVERITY_INFO:
021             return "information";
022         case mi::base::MESSAGE_SEVERITY_VERBOSE:
023             return "verbose";
024         case mi::base::MESSAGE_SEVERITY_DEBUG:
025             return "debug";
026         default:
027             return "unknown" ;
028     }
029 }
030 
031 // Define importer state, which is used in the importer function to carry additional data around
032 // for possible recursive invocations of importers. It contains a URI for the imported resource,
033 // a line number, and a pointer to a parent state supporting recursive imports.
034 class Vanilla_import_state
035     : public mi::base::Interface_implement< mi::neuraylib::IImpexp_state>
036 {
037     // Member variables to keep all necessary data
038     std::string               m_uri;
039     mi::Uint32                m_line_number;
040     const mi::neuraylib::IImpexp_state*  m_parent_state;
041 
042 public:
043     // Definition of all interface functions.
044 
045     const char* get_uri() const { return m_uri.empty() ? 0 : m_uri.c_str(); }
046 
047     mi::Uint32 get_line_number() const { return m_line_number; }
048 
049     void set_line_number( mi::Uint32 number) { m_line_number = number; }
050 
051     void incr_line_number() { ++m_line_number; }
052 
053     const mi::neuraylib::IImpexp_state* get_parent_state() const { return m_parent_state; }
054 
055     // Definition of corresponding setters / constructors. They are not part of the interface.
056 
057     // Default constructor, initializes line number to 1.
058     Vanilla_import_state() : m_line_number( 1), m_parent_state( 0) { }
059 
060     void set_uri( const char* uri) { m_uri = uri ? uri : ""; }
061 
062     void set_parent_state( const mi::neuraylib::IImpexp_state* parent_state)
063     {
064         m_parent_state = parent_state;
065     }
066 };
067 
068 // Define importer. It defines all meta information, for example, author, version numbers,
069 // which formats it supports etc. The longer format detection functions and import_elements
070 // function are implemented outside of the class body.
071 class Vanilla_importer
072     : public mi::base::Interface_implement< mi::neuraylib::IImporter>
073 {
074     mi::neuraylib::IPlugin_api* m_iplugin_api;
075 
076 public:
077     // Returns a state suitable for passing it to an import call.
078     // The parameters are used to initialize the corresponding properties of the state.
079     // The line number is set to 1.
080     mi::neuraylib::IImpexp_state* create_impexp_state(
081         const char* uri, const mi::neuraylib::IImpexp_state* parent_state) const
082     {
083         Vanilla_import_state* import_state = new Vanilla_import_state();
084         import_state->set_uri( uri);
085         import_state->set_parent_state( parent_state);
086         return import_state;
087     }
088 
089     // This importer supports the file name extensions ".vnl" and ".van".
090     const char* get_supported_extensions( mi::Uint32 i) const
091     {
092         switch( i) {
093             case 0:  return ".vnl";
094             case 1:  return ".van";
095             default: return 0;
096         }
097     }
098 
099     // Returns the confidence of the importer that its test_file_type() can identify the file and
100     // that the file format is fully supported.
101     mi::neuraylib::Impexp_priority get_priority() const
102     {
103         return mi::neuraylib::IMPEXP_PRIORITY_WELL_DEFINED;
104     }
105 
106     // Returns a concise single-line clear text description of the importer.
107     const char* get_name() const
108     {
109         return "mental images example vanilla (v1) importer";
110     }
111 
112     // Returns a concise single-line clear text description of the author of
113     // this importer.
114     const char* get_author() const
115     {
116         return "mental images GmbH, Berlin, Germany";
117     }
118 
119     // Returns the unique identifier for the importer.
120     mi::base::Uuid get_uuid() const
121     {
122         mi::base::Uuid uuid;
123         uuid.m_id1 = 0x338eca60;
124         uuid.m_id2 = 0x31004802;
125         uuid.m_id3 = 0xaab9046b;
126         uuid.m_id4 = 0x9e0b1d9b;
127         return uuid;
128     }
129 
130     // Returns the major version number of the importer.
131     mi::Uint32 get_major_version() const { return 1; }
132 
133     // Returns the minor version number of the importer.
134     mi::Uint32 get_minor_version() const { return 0; }
135 
136     // Returns true if the importer can handle the file type determined by the file name extension.
137     bool test_file_type( const char* extension) const;
138 
139     // Returns true if the importer can handle the file type determined by the file name extension
140     // and if the reader has sufficient capabilities for import.
141     bool test_file_type(
142         const char* extension, const mi::neuraylib::IReader* reader) const;
143 
144     // Imports all elements from the reader in a format determined by the file extension and
145     // (optionally) the lookahead of the reader.
146     mi::neuraylib::IImport_result* import_elements(
147         mi::neuraylib::ITransaction* transaction,
148         const char* extension,
149         mi::neuraylib::IReader* reader,
150         const mi::IMap* options,
151         mi::neuraylib::IImpexp_state* import_state) const;
152 
153     // Definition of constructors and support functions. They are not part of the interface.
154 
155     // Constructor.
156     Vanilla_importer( mi::neuraylib::IPlugin_api* iplugin_api)
157         : m_iplugin_api( iplugin_api)
158     {
159         m_iplugin_api->retain();
160     }
161 
162     // Destructor.
163     ~Vanilla_importer()
164     {
165         m_iplugin_api->release();
166     }
167 
168     // Format message with context and append it to the messages in the result.
169     static mi::neuraylib::IImport_result_ext* report_message(
170         mi::neuraylib::IImport_result_ext* result,
171         mi::Uint32 message_number,
172         mi::base::Message_severity message_severity,
173         std::string message,
174         const mi::neuraylib::IImpexp_state* import_state)
175     {
176         std::ostringstream s;
177         s << import_state->get_uri()
178           << ":" << import_state->get_line_number() << ": "
179           << "Vanilla importer message " << message_number << ", "
180           << "severity " << enum_to_str( message_severity) << ": "
181           << message;
182         // Report context of all parent import states from recursive
183         // invocations of import_elements in their own lines with indentation.
184         import_state = import_state->get_parent_state();
185         while( import_state) {
186             s << "\n    included from: " << import_state->get_uri()
187               << ":" << import_state->get_line_number();
188             import_state = import_state->get_parent_state();
189         }
190         result->message_push_back( message_number, message_severity, s.str().c_str());
191         return result;
192     }
193 };
194 
195 // Returns true if the importer can handle the file type determined by the file name extension.
196 bool Vanilla_importer::test_file_type( const char* extension) const
197 {
198     // This importer supports the file name extensions ".vnl" and ".van".
199     mi::Size len = std::strlen( extension);
200     return (len > 3)
201         &&  (( 0 == strcmp( extension + len - 4, ".vnl"))
202           || ( 0 == strcmp( extension + len - 4, ".van")));
203 }
204 
205 // Returns true if the importer can handle the file type determined by the file name extension
206 // and if the reader has sufficient capabilities for import.
207 bool Vanilla_importer::test_file_type(
208     const char* extension, const mi::neuraylib::IReader* reader) const
209 {
210     // Use magic header check if lookahead is available
211     if( reader->supports_lookahead()) {
212         // File has to start with "VANILLA" and linebreak, which can
213         // be \n or \r depending on the line ending convention in the file.
214         const char** buffer = 0;
215         mi::Sint64 n = reader->lookahead( 8, buffer);
216         return ( n >= 8) && (0 == std::strncmp( *buffer, "VANILLA", 7))
217                          && ((*buffer[7] == '\n') || (*buffer[7] == '\r'));
218     }
219     // This importer supports the file name extensions ".vnl" and ".van".
220     mi::Size len = std::strlen( extension);
221     return (len > 3)
222         &&  (( 0 == strcmp( extension + len - 4, ".vnl"))
223           || ( 0 == strcmp( extension + len - 4, ".van")));
224 }
225 
226 // Imports all elements from the reader in a format determined by the file extension and
227 // (optionally) the lookahead of the reader.
228 mi::neuraylib::IImport_result* Vanilla_importer::import_elements(
229     mi::neuraylib::ITransaction* transaction,
230     const char* extension,
231     mi::neuraylib::IReader* reader,
232     const mi::IMap* importer_options,
233     mi::neuraylib::IImpexp_state* import_state) const
234 {
235     // Create the importer result instance for the return value.
236     // If that fails something is really wrong and we return 0.
237     mi::neuraylib::IImport_result_ext* result
238         = transaction->create<mi::neuraylib::IImport_result_ext>( "Import_result_ext");
239     if( !result)
240         return 0;
241 
242     // Get the 'prefix' option.
243     MISTD::string prefix;
244     if( importer_options && importer_options->has_key( "prefix")) {
245         mi::base::Handle<const mi::IString> option(
246             importer_options->get_value<mi::IString>( "prefix"));
247         prefix = option->get_c_str();
248     }
249 
250     // Get the 'list_elements' option.
251     bool list_elements = false;
252     if( importer_options && importer_options->has_key( "list_elements")) {
253         mi::base::Handle<const mi::IBoolean> option(
254             importer_options->get_value<mi::IBoolean>( "list_elements"));
255         list_elements = option->get_value<bool>();
256     }
257 
258     // Before we start parsing the file, we create a group that collects all
259     // top-level elements. This will be our rootgroup.
260     std::string root_group_name = prefix + "Vanilla::root_group";
261     mi::base::Handle<mi::neuraylib::IGroup> rootgroup(
262         transaction->create<mi::neuraylib::IGroup>( "Group"));
263     mi::Sint32 error_code = transaction->store( rootgroup.get(), root_group_name.c_str());
264     if( error_code != 0)
265         return report_message( result, 4010, mi::base::MESSAGE_SEVERITY_ERROR,
266             "failed to create the root group", import_state);
267     rootgroup = transaction->edit<mi::neuraylib::IGroup>( root_group_name.c_str());
268 
269     // Register the rootgroup with the importer result.
270     result->set_rootgroup( root_group_name.c_str());
271 
272     // If the element list flag is set, record the rootgroup element also in the
273     // elements array of the result.
274     if( list_elements)
275         result->element_push_back( root_group_name.c_str());
276 
277     // Assume it is a line based text format and read it line by line.
278     // Assume lines are no longer than 256 chars, otherwise read it in pieces
279     char buffer[257];
280     while( reader->readline( buffer, 257) && buffer[0] != '\0') {
281 
282         // Periodically check whether the transaction is still open. If not, stop importing.
283         if( !transaction->is_open())
284             break;
285 
286         // Here you can process the buffer content of size len.
287         // It corresponds to the line import_state->get_line_number() of the input file.
288         mi::Size len = std::strlen( buffer);
289 
290         // We illustrate some actions triggered by fixed line numbers:
291         // Line 3 of a ".vnl" file triggers the recursive inclusion of the file "test2.van" file
292         // with a prefix.
293         mi::Size ext_len = std::strlen( extension);
294         if( 3 == import_state->get_line_number()
295              && (ext_len > 3)
296              && 0 == strcmp( extension + ext_len - 4, ".vnl")) {
297             // Get a IImport_api handle to call its import_elements() function
298             mi::base::Handle<mi::neuraylib::IImport_api> import_api(
299                 m_iplugin_api->get_api_component<mi::neuraylib::IImport_api>());
300             if( !import_api.is_valid_interface())
301                 // Error numbers from 4000 to 5999 are reserved for custom importer messages like
302                 // this one
303                 return report_message( result, 4001, mi::base::MESSAGE_SEVERITY_ERROR,
304                    "did not get a valid IImport_api object, import failed", import_state);
305             // Call the importer recursively to illustrate the handling of include files and
306             // similar things. We trigger this import only on a ".vnl" file and include the fixed
307             // file "test2.van". We give the included file an extra prefix "Prefix_".
308             mi::base::Handle<mi::neuraylib::IFactory> factory(
309                 m_iplugin_api->get_api_component<mi::neuraylib::IFactory>());
310             mi::base::Handle<mi::IString> child_prefix( factory->create<mi::IString>( "String"));
311             child_prefix->set_c_str( "Prefix_");
312             mi::base::Handle<mi::IMap> child_importer_options( factory->clone<mi::IMap>(
313                 importer_options));
314             child_importer_options->erase( "prefix");
315             child_importer_options->insert( "prefix", child_prefix.get());
316             mi::base::Handle<const mi::neuraylib::IImport_result> include_result(
317                 import_api->import_elements( transaction, "file:test2.van",
318                     child_importer_options.get(), import_state));
319             // Safety check, if this fails, the import is not continued.
320             if( !include_result.is_valid_interface())
321                 return report_message( result, 4002, mi::base::MESSAGE_SEVERITY_ERROR,
322                    "import was not able to create result object, import failed", import_state);
323             // Process the result. Even in the case of an error, we need to process the
324             // elements array.
325             if( list_elements)
326                 result->append_elements( include_result.get());
327             // Append messages of include to this result
328             result->append_messages( include_result.get());
329             // React on errors during processing of the include
330             if( include_result->get_error_number() > 0) {
331                 // Report the failure of the include as a separate message too
332                 report_message( result, 4003, mi::base::MESSAGE_SEVERITY_ERROR,
333                     "including file 'test2.van' failed.", import_state);
334             } else {
335                 // Recursive import was successful. The rootgroup of the
336                 // import is now appended to this rootgroup
337                 if( 0 == include_result->get_rootgroup())
338                     report_message( result, 4004, mi::base::MESSAGE_SEVERITY_ERROR,
339                         "include file 'test2.van' did not contain a rootgroup", import_state);
340                 else
341                     rootgroup->attach( include_result->get_rootgroup());
342             }
343         }
344 
345         // Line 4 triggers several messages and adds an empty group to the rootgroup.
346         if( 4 == import_state->get_line_number()) {
347             // Several messages, file parsing continues
348             report_message( result, 4005, mi::base::MESSAGE_SEVERITY_FATAL,
349                           "test message in line 4", import_state);
350             report_message( result, 4006, mi::base::MESSAGE_SEVERITY_ERROR,
351                           "test message in line 4", import_state);
352             report_message( result, 4007, mi::base::MESSAGE_SEVERITY_WARNING,
353                           "test message in line 4", import_state);
354             report_message( result, 4008, mi::base::MESSAGE_SEVERITY_INFO,
355                           "test message in line 4", import_state);
356             report_message( result, 4009, mi::base::MESSAGE_SEVERITY_VERBOSE,
357                           "test message in line 4", import_state);
358             // Create a group "Vanilla::Group1"
359             std::string group_name = prefix + "Vanilla::Group1";
360             mi::base::Handle<mi::neuraylib::IGroup> group(
361                 transaction->create<mi::neuraylib::IGroup>( "Group"));
362             mi::Sint32 error_code = transaction->store( group.get(), group_name.c_str());
363             if( error_code != 0)
364                 report_message( result, 4011, mi::base::MESSAGE_SEVERITY_ERROR,
365                     "unexpected error in line 4", import_state);
366             else {
367                 // Add this group to the rootgroup
368                 rootgroup->attach( group_name.c_str());
369                 // If get_list_elements_flag is set, record the new element in the elements array
370                 // of the result.
371                 if( list_elements)
372                     result->element_push_back( group_name.c_str());
373             }
374         }
375 
376         // Handle line numbers, buffer might end in '\n' or not if line was too long.
377         if( (len > 0) && ('\n' == buffer[len-1]))
378             import_state->incr_line_number();
379     }
380     if( reader->eof()) {
381         // Normal end
382         return result;
383     }
384     // Report error condition for a failed reader call
385     return report_message(
386         result,
387         static_cast<mi::Uint32>( reader->get_error_number()),
388         mi::base::MESSAGE_SEVERITY_ERROR,
389         reader->get_error_message() ? reader->get_error_message() : "",
390         import_state);
391 }

example_importer.cpp

001 /******************************************************************************
002  * © 1986, 2015 NVIDIA ARC GmbH. All rights reserved.
003  *****************************************************************************/
004 
005 // examples/example_importer.cpp
006 //
007 // Demonstrates the implementation of custom importers
008 
009 #include <iostream>
010 
011 // Include code shared by all examples.
012 #include "example_shared.h"
013 
014 // Include header file for the Vanilla importer.
015 #include "vanilla_importer.h"
016 
017 // The importer.
018 mi::base::Handle<mi::neuraylib::IImporter> importer;
019 
020 void configuration( mi::base::Handle<mi::neuraylib::INeuray> neuray)
021 {
022     // Register the Vanilla importer.
023     mi::base::Handle<mi::neuraylib::IExtension_api> extension_api(
024         neuray->get_api_component<mi::neuraylib::IExtension_api>());
025     check_success( extension_api.is_valid_interface());
026     mi::base::Handle<mi::neuraylib::IPlugin_api> plugin_api(
027         neuray->get_api_component<mi::neuraylib::IPlugin_api>());
028     check_success( plugin_api.is_valid_interface());
029     importer = new Vanilla_importer( plugin_api.get());
030     check_success( extension_api->register_importer( importer.get()) == 0);
031 }
032 
033 void test_importer( mi::base::Handle<mi::neuraylib::INeuray> neuray)
034 {
035     // Get the database, the global scope of the database, and create a transaction in the global
036     // scope.
037     mi::base::Handle<mi::neuraylib::IDatabase> database(
038         neuray->get_api_component<mi::neuraylib::IDatabase>());
039     check_success( database.is_valid_interface());
040     mi::base::Handle<mi::neuraylib::IScope> scope(
041         database->get_global_scope());
042     mi::base::Handle<mi::neuraylib::ITransaction> transaction(
043         scope->create_transaction());
044     check_success( transaction.is_valid_interface());
045 
046     // Prepare the importer options:
047     // We do not want to have an additional prefix, but a list of all imported elements.
048     mi::base::Handle<mi::IBoolean> list_elements( transaction->create<mi::IBoolean>( "Boolean"));
049     list_elements->set_value( true);
050     mi::base::Handle<mi::IMap> importer_options( transaction->create<mi::IMap>( "Map<Interface>"));
051     importer_options->insert( "list_elements", list_elements.get());
052 
053     // Import the file test1.vnl (implicitly using the Vanilla importer).
054     mi::base::Handle<mi::neuraylib::IImport_api> import_api(
055         neuray->get_api_component<mi::neuraylib::IImport_api>());
056     check_success( import_api.is_valid_interface());
057     mi::base::Handle<const mi::neuraylib::IImport_result> import_result(
058         import_api->import_elements( transaction.get(), "file:test1.vnl", importer_options.get()));
059     check_success( import_result.is_valid_interface());
060 
061     // Print all messages
062     for( mi::Size i = 0; i < import_result->get_messages_length(); ++i)
063         std::cout << import_result->get_message( i) << std::endl;
064 
065     // Print all imported elements
066     for( mi::Size i = 0; i < import_result->get_elements_length(); ++i)
067         std::cout << import_result->get_element( i) << std::endl;
068 
069     transaction->commit();
070 }
071 
072 int main( int /*argc*/, char* /*argv*/[])
073 {
074     // Access the neuray library
075     mi::base::Handle<mi::neuraylib::INeuray> neuray( load_and_get_ineuray());
076     check_success( neuray.is_valid_interface());
077 
078     // Configure the neuray library
079     configuration( neuray);
080 
081     // Start the neuray library
082     check_success( neuray->start() == 0);
083 
084     // Test the Vanilla importer
085     test_importer( neuray);
086 
087     // Shut down the neuray library
088     check_success( neuray->shutdown() == 0);
089 
090     // Unregister the Vanilla importer.
091     mi::base::Handle<mi::neuraylib::IExtension_api> extension_api(
092         neuray->get_api_component<mi::neuraylib::IExtension_api>());
093     check_success( extension_api->unregister_importer( importer.get()) == 0);
094     importer = 0;
095     extension_api = 0;
096     neuray = 0;
097 
098     // Unload the neuray library
099     check_success( unload());
100 
101     keep_console_open();
102     return EXIT_SUCCESS;
103 }