Amigos pude solucionar gracias a el aporte de Warkeys WK en un grupo de facebook, me compartio esta pagina: ¡https://www.mynotepaper.com/laravel-order-by-relationship-column-with-example!esta pagina
Route::get('/invoicesitems/{id}', function($id){
$invoiceItems = MasterInvoiceItem::select('*')
->where('id_load', '=', $id)
->with('variety')
->join('farms', 'master_invoice_items.id_farm', '=', 'farms.id')
->orderBy('farms.name', 'ASC')
->get();
return $invoiceItems;
});