Prior to 15.0, the results of a GROUP BY were sorted as if an ORDER BY had been done on the same columns. This sorting was a side effect of the method used to perform the GROUP BY aggregation rather than a deliberate feature of GROUP BY. 15.0 introduced new methods for performing aggregation which, though performing better, did not result in sorted output. Traceflag 450 causes ASE to use the pre-15.0 method in cases where the number of grouping columns is less than 32 (which was the limit on the number of GROUP BY columns).
The traceflag is intended as a temporary workaround until client code can be rewritten to perform an explicit ORDER BY where ordering is important.