🎉 Celebrating 25 Years of GameDev.net! 🎉

Not many can claim 25 years on the Internet! Join us in celebrating this milestone. Learn more about our history, and thank you for being a part of our community!

Calling Convention Bug?

Started by
1 comment, last by WitchLord 9 years, 2 months ago

I've discovered some sort of bug in the "CallSystemFunction." I have only encountered it in x64 Windows builds (Visual Studio 2013).

It looks like the code cleaning up the arguments incorrectly attempts to clean up the return variable. To trigger the bug, I register a class with a destructor and register a function that returns an instance of this object and takes one as an argument, so that the argument needs to be cleaned up. The cleanup code will then attempt to free the argument, but it will mistakenly pass a pointer to the return variable to be freed. This pointer is not a valid heap pointer, so there is a crash.

The attached file can be added to the test project to demonstrate the problem.

The fix may be something as simple as adding the following code the the cleanup section, but I am not familiar enough with how the AngelScript compiler manages its memory and sets up function calls to make that judgement.

as_callfunc.cpp(820)


// Clean up arguments
const asUINT cleanCount = sysFunc->cleanArgs.GetLength();
if( cleanCount )
{
    args = context->m_regs.stackPointer;
    if( callConv >= ICC_THISCALL )
        args += AS_PTR_SIZE;

    //** New code to address bug **/
    if( descr->DoesReturnOnStack() )
        args += AS_PTR_SIZE;

    asSSystemFunctionInterface::SClean *clean = sysFunc->cleanArgs.AddressOf();
    for( asUINT n = 0; n < cleanCount; n++, clean++ )
    {
        void **addr = (void**)&args[clean->off];
        if( clean->op == 0 )
        {
            if( *addr != 0 )
            {
                engine->CallObjectMethod(*addr, clean->ot->beh.release);
                *addr = 0;
            }
        }
        else 
        {
            asASSERT( clean->op == 1 || clean->op == 2 );
            asASSERT( *addr );

            if( clean->op == 2 )
                engine->CallObjectMethod(*addr, clean->ot->beh.destruct);
            
            engine->CallFree(*addr);
        }
    }
}
Advertisement

Thanks for the report. I'll look into it.

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

I've checked in the fix in revision 2162.

Thanks,

Andreas

AngelCode.com - game development and more - Reference DB - game developer references
AngelScript - free scripting library - BMFont - free bitmap font generator - Tower - free puzzle game

This topic is closed to new replies.

Advertisement