Awl - Asynchronous Work Library

[Ceylo]

Topic separated from http://www.sfml-dev.org/forum/viewtopic.php?p=38638#38638

A little note to tell you I stop working on sfeMovie for, in order to work on another library: Awl. This library should allow me and others to work with parallel programming much more easily, and sfeMovie will be able to benefit from it. I've indeed been a little bit fed up of all the parallel programming issues I got with sfeMovie.

The repository : https://github.com/Yalir/Awl
A sample with SFML : http://pastebin.com/4SKHyAms

Well, good luck to you! Sounds great! ;-)

I'm wondering, is the paste bin example the definitive syntax (i.e. AwlAsyncBlock/AwlMainThreadBlock and AwlCloseAsyncBlock/AwlCloseMainThreadBlock pairs) or will you (perhaps) change it later ?

Well.. yeah, I suppose this is the definitive syntax, and I can't find anything better as these are macros and thus need to be written as one word. Would you think of some other syntax ?

Moreover since yesterday there has been some additions and one now has:
AwlAsyncBlock
AwlCloseAsyncBlock
AwlCloseAsyncControlledBlock(taskRef) (gives back a Task object to allow cancelling/killing/waiting)

AwlAsyncCall(function)
AwlAsyncMethod(method, object)

AwlMainThreadBlock
AwlCloseAsyncMainThreadBlock (execute on main thread but don't wait for completion)
AwlCloseMainThreadBlock (execute on main thread and ensure completion before continuing)

AwlMainThreadCall(function)
AwlMainThreadMethod(method, object)

Hopefully I don't think there will be other macros like these, it seems enough to fit any situation.

Your syntax is not bad - you define a small DSL, that's it. However, if you want to keep the "C-like" syntax maybe the following could be a solution. I don't know exactly if it fits very well, though.

Code: [Select]

#define AwlAsyncBlock(functionBlock) \
{ \
    struct __awl_local_struct \
    { \
        static void __awl_async_block(void) \
        { \
            functionBlock \
        } \
    }; \
    awl::AsyncCall(boost::bind(__awl_local_struct::__awl_async_block)); \
}

:
:

AwlAsyncBlock
(
    tex1.LoadFromFile("big_image1.png");
    :
    :
)

So instead of using accolades you can use parenthesis.

And no more closing macro needed. I like it. Actually what I would have really loved would be writing the same with { } and no closing macro, for two reasons: to explicitly keep the idea of block, and to ease indentation.

As for the latest point, with { } the block is correctly indented, whereas with ( ) it's indented by one space and the closing parenthesis is not aligned to the opening one. These are details but annoying details for everyday's programming.

But.. I could successfully try the following syntax:

Code: [Select]

AwlAsyncBlock
({
    // some code
})

Which is more interesting than with the close macro. Thanks!

I'm not sure that you can have a ';' inside a macro argument, it would mess up the code parser.

And I strongly encourage you to open a new topic about Awl instead of "polluting" the sfeMovie thread

I'm not sure that you can have a ';' inside a macro argument, it would mess up the code parser.

Well, I just tested on gcc 4.2 and apple llvm compiler and both compiled the code fine. So apparently it works with these two at least.

[Ceylo]

According to my tests with LLVM it's also working fine, would need to test this with Visual Studio though.


Edit: it's working fine with Visual C++ too.

[posva]

Looks very nice. The code is short and understable.
What is the difference between this and sf::Threads?

[Ceylo]

The difference is it's short and understandable .
Try do to the same as the sample ( http://pastebin.com/4SKHyAms ) with sf::Thread and compare.

[posva]

I would have to create a fucntion and call it with a thread right?
By the way, is this the final version? even if you created it yesterday? xD

[Nexus]

By the way, I would rather use C++11 lamba expressions or normal functions than intransparent macros. You could also look at the threading features of the new standard library (std::async and Co.).

[Ceylo]

I would have to create a fucntion and call it with a thread right?
By the way, is this the final version? even if you created it yesterday? xD

Indeed you'd have to create a thread (thus another function), launch it, take care of the thread's life scope, and find how to execute the AwlMainThreadBlock on the main thread.

Edit: and for sure no, this isn't the final version, there's still much work to do.

By the way, I would rather use C++11 lamba expressions or normal functions than intransparent macros. You could also look at the threading features of the new standard library (std::async and Co.).

Well, I don't know how well the C++11 standard is supported but I want Awl to be usable by most today's compilers so I don't think this is possible. Moreover, even if the macro is intransparent, it fits my needs. The user shouldn't have to know how it works and it's ok. Thus transparency is.. to me, a detail.

[Hiura]

I want Awl to be usable by most today's compilers so I don't think this is possible.

You're absolutely right. Lambda expression are great, and so do most C++11 features, but only a few compilers support them. Here is a page stating the support progress of most compilers : http://wiki.apache.org/stdcxx/C++0xCompilerSupport

[Ceylo]

Looking for a way to provide input values to the asynchronous blocks, I've found a little bit more about what can and what can't be done with macros, especially as for the parser issues.

See:

Code: [Select]


{
    function1(arg);                    // 1. Ok
    function2(arg1, arg2);             // 2. Ok
    free(something), something = NULL; // 3. Not ok
}
What I found out is that the preprocessor doesn't get confused if the "," is inside parenthesis, like in a function call, but otherwise it does.

[Laurent]

Indeed. That's why complex macro arguments are often inside an extra pair of parenthesis:

Code: [Select]

#define MY_MACRO(args) ...

void f()
{
    MY_MACRO((...args...));
}

[Nexus]

You can use variadic macros to treat a comma-separated list like a single argument via __VA_ARGS__. This language feature is not part of standard C++98 however.

[Ceylo]

I'm currently trying to define a macro able to take input arguments to be passed to the asynchronous block. For now I have the following, with 'input' as a list of pairs, and 'blockCode' that is the code in the block (hehe ).

For now I would like to be able to write the following:

Code: [Select]

AwlAsyncBlock
({Awl(var1), Awl(var2)}, // define input parameters here
{
    // some code
})

Or ideally:

Code: [Select]

AwlAsyncBlock
({var1, var2}, // define input parameters here
{
    // some code
})

Code: [Select]

#define Awl(var) std::pair<std::string, void *>(#var, (void *)&var)
#define AwlInputEnd std::pair<std::string, void *>("__awl_null", NULL)

Code: [Select]

{
std::pair<std::string, void *> items[] = input;
std::map <std::string, void *> input_values;
for (int i = 0; items[i] != AwlInputEnd; i++)
input_values[items[i].first] = items[i].second;

struct local_struct
{
static void async_block(awl::Task *self)
{
blockCode
}
};
awl::Callback f(boost::bind(local_struct::async_block, _1));
awl::TaskRef t(new awl::Task(f));
t->SetInput(input_values);
awl::ThreadPool::Default().ScheduleTaskForExecution(t);
}

Which is not possible because of the comma issue. And if I put parenthesis around it, it is no more considered an array and std::pair<std::string, void *> items[] = input; does no more work.

Because of this issue, I'm rather looking for a way to write something like:

Code: [Select]

AwlAsyncBlock
(AwlInput({var1, var2}), // define input parameters here
{
    // some code
})
Which I don't know how to do yet.


Now as for the variadic macro, I've been taking a look at it. I know it's part of C99 standard which is ok to me. But I still can't get it to work. I would like to build a map of the input parameters where the key is the variable name and the associated value a pointer to that variable. That's why I'm defining Awl(var) as std::pair<std::string, void *>(#var, (void *)&var), and I would like to store these pairs in the input map. But the point with variable argument lists is you lose the name of the original variable name. You just get a list of items.

[Ceylo]

I could finally get something rather awful to work:

Code: [Select]

std::string some_text = "tested!!!";
AwlAsyncBlock3
(AwlInput(Awl(some_text), NULL)
 ,
 {
     std::string& some_text = *(std::string *)self->input["some_text"];
     std::cout << some_text << std::endl;
})

Here you can add as much input parameters as you want, but it's far from easy to use...

To do so I used variadic macro, wrapping in pairs of <string key, void *value> and <void *> casting/uncasting. But at least it's working, I hope to find some better way of writing it.

[Ceylo]

After a bit of thinking I've got something much cleaner working

Code: [Select]

std::string some_text = "testing";
AwlAsyncBlock_1
(std::string, some_text,
 {
     std::cout << some_text << std::endl;
 })

Here the trick is getting rid of any variable argument count and using the macro to redeclare the exported variable as a reference to the original one. Thus I need one macro for each parameter count (here 1 parameter) like boost does, but at least it's .. well.. I like it.

[Ceylo]

Ok.. I realised there are a few constraints with this way : you can't pass raw arrays as raw arrays (have to go with a classic pointer) and I cannot give the user the ability to choose how to give the parameter (by reference or value).

Thus.. I'm wondering what's the best solution. Stick with parameters by value, and force the user to use pointers if he wants references, or stick with references (not pointer, although permitted) only.

Plus the user can only give variables, but no direct value like in function(14).


Edit: I think I'll stick with references as.. when declaring an inner block one is rather expecting to work on the same variables than on copies.

Edit 2: changes pushed.